2023-09-29T23:18:07.918 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-29T23:18:07.918 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568 --owner scheduled_yuriw@teuthology --verbose --description rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/failover} -- /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568/orig.config.yaml 2023-09-29T23:18:07.949 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T23:18:08.031 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568 branch: wip-yuri4-testing-2023-09-29-0928-quincy description: rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/failover} email: yweinste@redhat.com first_in_suite: false job_id: '7406568' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 2 size: 30 os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-yuri4-testing-2023-09-29-0928-quincy ceph: conf: client: debug client: 20 debug mgrc: 20 debug ms: 1 global: mon election default strategy: 3 mds: debug mgrc: 20 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mgr ttl cache expire seconds: 5 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: snappy bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug mgrc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri4-testing-2023-09-29-0928-quincy sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mgr.x - mon.a - mon.c - mds.a - mds.c - osd.0 - client.0 - - mgr.y - mgr.z - mon.b - mds.b - osd.1 - osd.2 - osd.3 - client.1 seed: 9810 sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri4-testing-2023-09-29-0928-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 targets: smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: log-ignorelist: - overall HEALTH_ - \(MGR_DOWN\) - \(PG_ - replacing it with standby - No standby daemons available wait-for-scrub: false - cephfs_test_runner: modules: - tasks.mgr.test_failover teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-29_19:38:56 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-29T23:18:08.032 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa; will attempt to use it 2023-09-29T23:18:08.032 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks 2023-09-29T23:18:08.033 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-29T23:18:08.034 INFO:teuthology.task.internal:Checking packages... 2023-09-29T23:18:08.055 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '93dce5e17918f8aca65993a7a2dc40233a47b481' 2023-09-29T23:18:08.055 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-29T23:18:08.056 INFO:teuthology.packaging:ref: None 2023-09-29T23:18:08.056 INFO:teuthology.packaging:tag: None 2023-09-29T23:18:08.056 INFO:teuthology.packaging:branch: wip-yuri4-testing-2023-09-29-0928-quincy 2023-09-29T23:18:08.056 INFO:teuthology.packaging:sha1: 93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-29T23:18:08.056 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-yuri4-testing-2023-09-29-0928-quincy 2023-09-29T23:18:08.271 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1165-g93dce5e1-1focal 2023-09-29T23:18:08.272 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-29T23:18:08.281 INFO:teuthology.task.internal:no buildpackages task found 2023-09-29T23:18:08.281 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-29T23:18:08.301 INFO:teuthology.task.internal:Saving configuration 2023-09-29T23:18:08.313 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-29T23:18:08.321 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-29T23:18:08.346 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-29 23:11:53.389423', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-09-29T23:18:08.371 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-29 23:11:53.390738', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2023-09-29T23:18:08.371 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-29T23:18:08.378 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'] 2023-09-29T23:18:08.378 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1'] 2023-09-29T23:18:08.378 INFO:teuthology.run_tasks:Running task console_log... 2023-09-29T23:18:08.452 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fbefba81040>, signals=[15]) 2023-09-29T23:18:08.453 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-29T23:18:08.460 INFO:teuthology.task.internal:Opening connections... 2023-09-29T23:18:08.461 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T23:18:08.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:18:08.512 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2023-09-29T23:18:08.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:18:08.561 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-29T23:18:08.567 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2023-09-29T23:18:08.593 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2023-09-29T23:18:08.593 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2023-09-29T23:18:08.640 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="Ubuntu" 2023-09-29T23:18:08.640 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-09-29T23:18:08.641 INFO:teuthology.orchestra.run.smithi143.stdout:ID=ubuntu 2023-09-29T23:18:08.641 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE=debian 2023-09-29T23:18:08.641 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-09-29T23:18:08.641 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="20.04" 2023-09-29T23:18:08.641 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-29T23:18:08.641 INFO:teuthology.orchestra.run.smithi143.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-29T23:18:08.641 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-29T23:18:08.641 INFO:teuthology.orchestra.run.smithi143.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-29T23:18:08.641 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_CODENAME=focal 2023-09-29T23:18:08.641 INFO:teuthology.orchestra.run.smithi143.stdout:UBUNTU_CODENAME=focal 2023-09-29T23:18:08.642 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2023-09-29T23:18:08.665 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2023-09-29T23:18:08.672 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2023-09-29T23:18:08.672 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-09-29T23:18:08.678 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Ubuntu" 2023-09-29T23:18:08.678 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2023-09-29T23:18:08.679 INFO:teuthology.orchestra.run.smithi156.stdout:ID=ubuntu 2023-09-29T23:18:08.679 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE=debian 2023-09-29T23:18:08.679 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2023-09-29T23:18:08.679 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="20.04" 2023-09-29T23:18:08.679 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.ubuntu.com/" 2023-09-29T23:18:08.679 INFO:teuthology.orchestra.run.smithi156.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-09-29T23:18:08.680 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-09-29T23:18:08.680 INFO:teuthology.orchestra.run.smithi156.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-09-29T23:18:08.680 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_CODENAME=focal 2023-09-29T23:18:08.680 INFO:teuthology.orchestra.run.smithi156.stdout:UBUNTU_CODENAME=focal 2023-09-29T23:18:08.681 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-09-29T23:18:08.698 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-29T23:18:08.709 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-29T23:18:08.716 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-29T23:18:08.717 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2023-09-29T23:18:08.719 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2023-09-29T23:18:08.725 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-29T23:18:08.732 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-29T23:18:08.733 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2023-09-29T23:18:08.735 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2023-09-29T23:18:08.743 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-29T23:18:08.817 INFO:teuthology.run_tasks:Running task kernel... 2023-09-29T23:18:08.833 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-09-29T23:18:08.833 INFO:teuthology.task.kernel:config {'mgr.x': {'kdb': True, 'sha1': 'distro'}, 'mgr.y': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-09-29T23:18:08.833 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-29T23:18:08.833 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-29T23:18:08.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T23:18:08.840 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2023-09-29T23:18:08.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T23:18:08.841 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2023-09-29T23:18:08.847 INFO:teuthology.orchestra.run.smithi143.stdout:5.4.0-124-generic 2023-09-29T23:18:08.847 INFO:teuthology.task.kernel:Running kernel on smithi143: 5.4.0-124-generic 2023-09-29T23:18:08.847 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get clean 2023-09-29T23:18:08.890 INFO:teuthology.orchestra.run.smithi156.stdout:5.4.0-124-generic 2023-09-29T23:18:08.890 INFO:teuthology.task.kernel:Running kernel on smithi156: 5.4.0-124-generic 2023-09-29T23:18:08.891 DEBUG:teuthology.orchestra.run.smithi156:> sudo apt-get clean 2023-09-29T23:18:08.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo apt-get update 2023-09-29T23:18:09.019 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get update 2023-09-29T23:18:09.159 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-29T23:18:09.160 INFO:teuthology.orchestra.run.smithi156.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-09-29T23:18:09.160 INFO:teuthology.orchestra.run.smithi156.stdout:Get:3 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-09-29T23:18:09.164 INFO:teuthology.orchestra.run.smithi143.stdout:Get:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB] 2023-09-29T23:18:09.230 INFO:teuthology.orchestra.run.smithi156.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-09-29T23:18:09.275 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-29T23:18:09.357 INFO:teuthology.orchestra.run.smithi143.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease [114 kB] 2023-09-29T23:18:09.437 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [652 kB] 2023-09-29T23:18:09.518 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,464 kB] 2023-09-29T23:18:09.550 INFO:teuthology.orchestra.run.smithi156.stdout:Get:5 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [652 kB] 2023-09-29T23:18:09.563 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [385 kB] 2023-09-29T23:18:09.563 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-09-29T23:18:09.564 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,218 kB] 2023-09-29T23:18:09.576 INFO:teuthology.orchestra.run.smithi143.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-09-29T23:18:09.577 INFO:teuthology.orchestra.run.smithi143.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [309 kB] 2023-09-29T23:18:09.579 INFO:teuthology.orchestra.run.smithi143.stdout:Get:11 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-09-29T23:18:09.579 INFO:teuthology.orchestra.run.smithi143.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [615 kB] 2023-09-29T23:18:09.583 INFO:teuthology.orchestra.run.smithi143.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [886 kB] 2023-09-29T23:18:09.589 INFO:teuthology.orchestra.run.smithi143.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [185 kB] 2023-09-29T23:18:09.590 INFO:teuthology.orchestra.run.smithi143.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-09-29T23:18:09.590 INFO:teuthology.orchestra.run.smithi143.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-09-29T23:18:09.591 INFO:teuthology.orchestra.run.smithi143.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-09-29T23:18:09.598 INFO:teuthology.orchestra.run.smithi143.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-09-29T23:18:09.598 INFO:teuthology.orchestra.run.smithi143.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-09-29T23:18:09.632 INFO:teuthology.orchestra.run.smithi156.stdout:Get:6 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [2,464 kB] 2023-09-29T23:18:09.670 INFO:teuthology.orchestra.run.smithi156.stdout:Get:7 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [385 kB] 2023-09-29T23:18:09.673 INFO:teuthology.orchestra.run.smithi156.stdout:Get:8 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [13.2 kB] 2023-09-29T23:18:09.673 INFO:teuthology.orchestra.run.smithi156.stdout:Get:9 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [33.3 kB] 2023-09-29T23:18:09.674 INFO:teuthology.orchestra.run.smithi156.stdout:Get:10 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [2,218 kB] 2023-09-29T23:18:09.684 INFO:teuthology.orchestra.run.smithi156.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,852 kB] 2023-09-29T23:18:09.690 INFO:teuthology.orchestra.run.smithi156.stdout:Get:12 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [309 kB] 2023-09-29T23:18:09.694 INFO:teuthology.orchestra.run.smithi156.stdout:Get:13 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [572 B] 2023-09-29T23:18:09.694 INFO:teuthology.orchestra.run.smithi156.stdout:Get:14 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [886 kB] 2023-09-29T23:18:09.700 INFO:teuthology.orchestra.run.smithi156.stdout:Get:15 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [615 kB] 2023-09-29T23:18:09.702 INFO:teuthology.orchestra.run.smithi143.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-backports InRelease [108 kB] 2023-09-29T23:18:09.706 INFO:teuthology.orchestra.run.smithi156.stdout:Get:16 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [185 kB] 2023-09-29T23:18:09.707 INFO:teuthology.orchestra.run.smithi156.stdout:Get:17 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [19.1 kB] 2023-09-29T23:18:09.707 INFO:teuthology.orchestra.run.smithi156.stdout:Get:18 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,196 B] 2023-09-29T23:18:09.708 INFO:teuthology.orchestra.run.smithi156.stdout:Get:19 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [23.6 kB] 2023-09-29T23:18:09.708 INFO:teuthology.orchestra.run.smithi156.stdout:Get:20 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,504 B] 2023-09-29T23:18:09.722 INFO:teuthology.orchestra.run.smithi156.stdout:Get:21 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [548 B] 2023-09-29T23:18:09.780 INFO:teuthology.orchestra.run.smithi156.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [885 kB] 2023-09-29T23:18:09.790 INFO:teuthology.orchestra.run.smithi156.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [469 kB] 2023-09-29T23:18:09.795 INFO:teuthology.orchestra.run.smithi156.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-09-29T23:18:09.796 INFO:teuthology.orchestra.run.smithi156.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,330 kB] 2023-09-29T23:18:09.812 INFO:teuthology.orchestra.run.smithi156.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.7 kB] 2023-09-29T23:18:09.812 INFO:teuthology.orchestra.run.smithi156.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [325 kB] 2023-09-29T23:18:09.814 INFO:teuthology.orchestra.run.smithi156.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-09-29T23:18:09.815 INFO:teuthology.orchestra.run.smithi156.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [748 kB] 2023-09-29T23:18:09.820 INFO:teuthology.orchestra.run.smithi156.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,117 kB] 2023-09-29T23:18:09.827 INFO:teuthology.orchestra.run.smithi156.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [267 kB] 2023-09-29T23:18:09.830 INFO:teuthology.orchestra.run.smithi156.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-09-29T23:18:09.830 INFO:teuthology.orchestra.run.smithi156.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.9 kB] 2023-09-29T23:18:09.830 INFO:teuthology.orchestra.run.smithi156.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-09-29T23:18:09.842 INFO:teuthology.orchestra.run.smithi156.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-09-29T23:18:09.842 INFO:teuthology.orchestra.run.smithi156.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-09-29T23:18:09.843 INFO:teuthology.orchestra.run.smithi156.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-09-29T23:18:09.843 INFO:teuthology.orchestra.run.smithi156.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-09-29T23:18:09.843 INFO:teuthology.orchestra.run.smithi156.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-09-29T23:18:09.844 INFO:teuthology.orchestra.run.smithi143.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [885 kB] 2023-09-29T23:18:09.859 INFO:teuthology.orchestra.run.smithi156.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-09-29T23:18:09.859 INFO:teuthology.orchestra.run.smithi156.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-09-29T23:18:09.860 INFO:teuthology.orchestra.run.smithi156.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-09-29T23:18:10.090 INFO:teuthology.orchestra.run.smithi143.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [2,852 kB] 2023-09-29T23:18:10.253 INFO:teuthology.orchestra.run.smithi143.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [469 kB] 2023-09-29T23:18:10.265 INFO:teuthology.orchestra.run.smithi143.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.1 kB] 2023-09-29T23:18:10.266 INFO:teuthology.orchestra.run.smithi143.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [2,330 kB] 2023-09-29T23:18:10.326 INFO:teuthology.orchestra.run.smithi143.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [34.7 kB] 2023-09-29T23:18:10.326 INFO:teuthology.orchestra.run.smithi143.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [325 kB] 2023-09-29T23:18:10.347 INFO:teuthology.orchestra.run.smithi143.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [572 B] 2023-09-29T23:18:10.511 INFO:teuthology.orchestra.run.smithi143.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,117 kB] 2023-09-29T23:18:11.145 INFO:teuthology.orchestra.run.smithi143.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [748 kB] 2023-09-29T23:18:11.182 INFO:teuthology.orchestra.run.smithi143.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [267 kB] 2023-09-29T23:18:11.195 INFO:teuthology.orchestra.run.smithi143.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [25.6 kB] 2023-09-29T23:18:11.196 INFO:teuthology.orchestra.run.smithi143.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,436 B] 2023-09-29T23:18:11.197 INFO:teuthology.orchestra.run.smithi143.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [25.9 kB] 2023-09-29T23:18:11.198 INFO:teuthology.orchestra.run.smithi143.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,484 B] 2023-09-29T23:18:11.199 INFO:teuthology.orchestra.run.smithi143.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [620 B] 2023-09-29T23:18:11.199 INFO:teuthology.orchestra.run.smithi143.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2023-09-29T23:18:11.200 INFO:teuthology.orchestra.run.smithi143.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2023-09-29T23:18:11.226 INFO:teuthology.orchestra.run.smithi143.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2023-09-29T23:18:11.390 INFO:teuthology.orchestra.run.smithi143.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2023-09-29T23:18:11.574 INFO:teuthology.orchestra.run.smithi143.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2023-09-29T23:18:11.596 INFO:teuthology.orchestra.run.smithi143.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2023-09-29T23:18:13.520 INFO:teuthology.orchestra.run.smithi156.stdout:Fetched 17.4 MB in 2s (7,043 kB/s) 2023-09-29T23:18:13.717 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 17.4 MB in 3s (6,928 kB/s) 2023-09-29T23:18:14.927 INFO:teuthology.orchestra.run.smithi156.stdout:Reading package lists... 2023-09-29T23:18:14.955 DEBUG:teuthology.orchestra.run.smithi156:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-29T23:18:15.050 INFO:teuthology.orchestra.run.smithi156.stdout:Reading package lists... 2023-09-29T23:18:15.146 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2023-09-29T23:18:15.172 DEBUG:teuthology.orchestra.run.smithi143:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-29T23:18:15.238 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2023-09-29T23:18:15.252 INFO:teuthology.orchestra.run.smithi156.stdout:Building dependency tree... 2023-09-29T23:18:15.253 INFO:teuthology.orchestra.run.smithi156.stdout:Reading state information... 2023-09-29T23:18:15.440 INFO:teuthology.orchestra.run.smithi143.stdout:Building dependency tree... 2023-09-29T23:18:15.441 INFO:teuthology.orchestra.run.smithi143.stdout:Reading state information... 2023-09-29T23:18:15.445 INFO:teuthology.orchestra.run.smithi156.stdout:The following additional packages will be installed: 2023-09-29T23:18:15.446 INFO:teuthology.orchestra.run.smithi156.stdout: linux-generic linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-29T23:18:15.446 INFO:teuthology.orchestra.run.smithi156.stdout: linux-headers-generic linux-image-5.4.0-163-generic 2023-09-29T23:18:15.446 INFO:teuthology.orchestra.run.smithi156.stdout: linux-modules-5.4.0-163-generic linux-modules-extra-5.4.0-163-generic 2023-09-29T23:18:15.447 INFO:teuthology.orchestra.run.smithi156.stdout:Suggested packages: 2023-09-29T23:18:15.448 INFO:teuthology.orchestra.run.smithi156.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-09-29T23:18:15.448 INFO:teuthology.orchestra.run.smithi156.stdout:Recommended packages: 2023-09-29T23:18:15.448 INFO:teuthology.orchestra.run.smithi156.stdout: thermald 2023-09-29T23:18:15.502 INFO:teuthology.orchestra.run.smithi156.stdout:The following NEW packages will be installed: 2023-09-29T23:18:15.502 INFO:teuthology.orchestra.run.smithi156.stdout: linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-29T23:18:15.502 INFO:teuthology.orchestra.run.smithi156.stdout: linux-image-5.4.0-163-generic linux-modules-5.4.0-163-generic 2023-09-29T23:18:15.502 INFO:teuthology.orchestra.run.smithi156.stdout: linux-modules-extra-5.4.0-163-generic 2023-09-29T23:18:15.503 INFO:teuthology.orchestra.run.smithi156.stdout:The following packages will be upgraded: 2023-09-29T23:18:15.504 INFO:teuthology.orchestra.run.smithi156.stdout: linux-generic linux-headers-generic linux-image-generic 2023-09-29T23:18:15.555 INFO:teuthology.orchestra.run.smithi156.stdout:3 upgraded, 5 newly installed, 0 to remove and 263 not upgraded. 2023-09-29T23:18:15.555 INFO:teuthology.orchestra.run.smithi156.stdout:Need to get 77.1 MB of archives. 2023-09-29T23:18:15.555 INFO:teuthology.orchestra.run.smithi156.stdout:After this operation, 380 MB of additional disk space will be used. 2023-09-29T23:18:15.555 INFO:teuthology.orchestra.run.smithi156.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-163-generic amd64 5.4.0-163.180 [15.0 MB] 2023-09-29T23:18:15.619 INFO:teuthology.orchestra.run.smithi143.stdout:The following additional packages will be installed: 2023-09-29T23:18:15.619 INFO:teuthology.orchestra.run.smithi143.stdout: linux-generic linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-29T23:18:15.619 INFO:teuthology.orchestra.run.smithi143.stdout: linux-headers-generic linux-image-5.4.0-163-generic 2023-09-29T23:18:15.620 INFO:teuthology.orchestra.run.smithi143.stdout: linux-modules-5.4.0-163-generic linux-modules-extra-5.4.0-163-generic 2023-09-29T23:18:15.620 INFO:teuthology.orchestra.run.smithi143.stdout:Suggested packages: 2023-09-29T23:18:15.620 INFO:teuthology.orchestra.run.smithi143.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2023-09-29T23:18:15.620 INFO:teuthology.orchestra.run.smithi143.stdout:Recommended packages: 2023-09-29T23:18:15.621 INFO:teuthology.orchestra.run.smithi143.stdout: thermald 2023-09-29T23:18:15.684 INFO:teuthology.orchestra.run.smithi143.stdout:The following NEW packages will be installed: 2023-09-29T23:18:15.685 INFO:teuthology.orchestra.run.smithi143.stdout: linux-headers-5.4.0-163 linux-headers-5.4.0-163-generic 2023-09-29T23:18:15.685 INFO:teuthology.orchestra.run.smithi143.stdout: linux-image-5.4.0-163-generic linux-modules-5.4.0-163-generic 2023-09-29T23:18:15.685 INFO:teuthology.orchestra.run.smithi143.stdout: linux-modules-extra-5.4.0-163-generic 2023-09-29T23:18:15.686 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages will be upgraded: 2023-09-29T23:18:15.686 INFO:teuthology.orchestra.run.smithi143.stdout: linux-generic linux-headers-generic linux-image-generic 2023-09-29T23:18:15.738 INFO:teuthology.orchestra.run.smithi143.stdout:3 upgraded, 5 newly installed, 0 to remove and 263 not upgraded. 2023-09-29T23:18:15.739 INFO:teuthology.orchestra.run.smithi143.stdout:Need to get 77.1 MB of archives. 2023-09-29T23:18:15.739 INFO:teuthology.orchestra.run.smithi143.stdout:After this operation, 380 MB of additional disk space will be used. 2023-09-29T23:18:15.739 INFO:teuthology.orchestra.run.smithi143.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-163-generic amd64 5.4.0-163.180 [15.0 MB] 2023-09-29T23:18:15.895 INFO:teuthology.orchestra.run.smithi156.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-163-generic amd64 5.4.0-163.180 [10.5 MB] 2023-09-29T23:18:15.991 INFO:teuthology.orchestra.run.smithi156.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-163-generic amd64 5.4.0-163.180 [39.2 MB] 2023-09-29T23:18:16.013 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-163-generic amd64 5.4.0-163.180 [10.5 MB] 2023-09-29T23:18:16.103 INFO:teuthology.orchestra.run.smithi143.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-163-generic amd64 5.4.0-163.180 [39.2 MB] 2023-09-29T23:18:16.460 INFO:teuthology.orchestra.run.smithi156.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.163.160 [1,904 B] 2023-09-29T23:18:16.460 INFO:teuthology.orchestra.run.smithi156.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.163.160 [2,604 B] 2023-09-29T23:18:16.461 INFO:teuthology.orchestra.run.smithi156.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163 all 5.4.0-163.180 [11.0 MB] 2023-09-29T23:18:16.553 INFO:teuthology.orchestra.run.smithi156.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163-generic amd64 5.4.0-163.180 [1,358 kB] 2023-09-29T23:18:16.563 INFO:teuthology.orchestra.run.smithi156.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.163.160 [2,468 B] 2023-09-29T23:18:16.569 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.163.160 [1,904 B] 2023-09-29T23:18:16.569 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.163.160 [2,604 B] 2023-09-29T23:18:16.570 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163 all 5.4.0-163.180 [11.0 MB] 2023-09-29T23:18:16.663 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-163-generic amd64 5.4.0-163.180 [1,358 kB] 2023-09-29T23:18:16.673 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.163.160 [2,468 B] 2023-09-29T23:18:17.176 INFO:teuthology.orchestra.run.smithi156.stdout:Fetched 77.1 MB in 1s (73.2 MB/s) 2023-09-29T23:18:17.384 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package linux-modules-5.4.0-163-generic. 2023-09-29T23:18:17.390 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 77.1 MB in 1s (78.6 MB/s) 2023-09-29T23:18:17.581 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-modules-5.4.0-163-generic. 2023-09-29T23:18:19.106 INFO:teuthology.orchestra.run.smithi156.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-09-29T23:18:19.111 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../0-linux-modules-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-29T23:18:19.181 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:19.297 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2023-09-29T23:18:19.302 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../0-linux-modules-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-29T23:18:19.378 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:21.098 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package linux-image-5.4.0-163-generic. 2023-09-29T23:18:21.114 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../1-linux-image-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-29T23:18:21.214 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:21.327 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-image-5.4.0-163-generic. 2023-09-29T23:18:21.340 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../1-linux-image-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-29T23:18:21.456 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:22.251 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-163-generic. 2023-09-29T23:18:22.265 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-29T23:18:22.309 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:22.529 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-163-generic. 2023-09-29T23:18:22.543 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-29T23:18:22.588 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:27.228 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../3-linux-generic_5.4.0.163.160_amd64.deb ... 2023-09-29T23:18:27.354 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking linux-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-29T23:18:27.355 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../3-linux-generic_5.4.0.163.160_amd64.deb ... 2023-09-29T23:18:27.480 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-29T23:18:27.730 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.163.160_amd64.deb ... 2023-09-29T23:18:27.798 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.163.160_amd64.deb ... 2023-09-29T23:18:27.858 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking linux-image-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-29T23:18:27.967 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-image-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-29T23:18:28.202 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package linux-headers-5.4.0-163. 2023-09-29T23:18:28.216 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../5-linux-headers-5.4.0-163_5.4.0-163.180_all.deb ... 2023-09-29T23:18:28.252 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-29T23:18:28.294 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-headers-5.4.0-163. 2023-09-29T23:18:28.309 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../5-linux-headers-5.4.0-163_5.4.0-163.180_all.deb ... 2023-09-29T23:18:28.353 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-29T23:18:33.552 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package linux-headers-5.4.0-163-generic. 2023-09-29T23:18:33.575 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../6-linux-headers-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-29T23:18:33.618 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:33.629 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package linux-headers-5.4.0-163-generic. 2023-09-29T23:18:33.649 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../6-linux-headers-5.4.0-163-generic_5.4.0-163.180_amd64.deb ... 2023-09-29T23:18:33.687 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:36.290 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.163.160_amd64.deb ... 2023-09-29T23:18:36.335 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.163.160_amd64.deb ... 2023-09-29T23:18:36.395 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking linux-headers-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-29T23:18:36.580 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking linux-headers-generic (5.4.0.163.160) over (5.4.0.124.125) ... 2023-09-29T23:18:36.729 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:36.915 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up linux-modules-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:37.927 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-29T23:18:38.070 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:38.161 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up linux-headers-5.4.0-163 (5.4.0-163.180) ... 2023-09-29T23:18:38.287 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:39.227 INFO:teuthology.orchestra.run.smithi143.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-163-generic 2023-09-29T23:18:39.239 INFO:teuthology.orchestra.run.smithi143.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-163-generic 2023-09-29T23:18:39.364 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:39.440 INFO:teuthology.orchestra.run.smithi156.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-163-generic 2023-09-29T23:18:39.440 INFO:teuthology.orchestra.run.smithi156.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-163-generic 2023-09-29T23:18:39.465 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:39.572 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up linux-headers-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:39.707 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up linux-modules-extra-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:40.650 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-image-generic (5.4.0.163.160) ... 2023-09-29T23:18:40.776 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-headers-generic (5.4.0.163.160) ... 2023-09-29T23:18:40.891 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up linux-image-generic (5.4.0.163.160) ... 2023-09-29T23:18:40.902 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up linux-generic (5.4.0.163.160) ... 2023-09-29T23:18:41.020 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:41.025 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up linux-headers-generic (5.4.0.163.160) ... 2023-09-29T23:18:41.087 INFO:teuthology.orchestra.run.smithi143.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-09-29T23:18:41.087 INFO:teuthology.orchestra.run.smithi143.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-163-generic 2023-09-29T23:18:41.135 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up linux-generic (5.4.0.163.160) ... 2023-09-29T23:18:41.262 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for linux-image-5.4.0-163-generic (5.4.0-163.180) ... 2023-09-29T23:18:41.331 INFO:teuthology.orchestra.run.smithi156.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-09-29T23:18:41.332 INFO:teuthology.orchestra.run.smithi156.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-163-generic 2023-09-29T23:19:03.235 INFO:teuthology.orchestra.run.smithi143.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-09-29T23:19:03.235 INFO:teuthology.orchestra.run.smithi143.stdout:Sourcing file `/etc/default/grub' 2023-09-29T23:19:03.243 INFO:teuthology.orchestra.run.smithi143.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-29T23:19:03.251 INFO:teuthology.orchestra.run.smithi143.stdout:Generating grub configuration file ... 2023-09-29T23:19:03.476 INFO:teuthology.orchestra.run.smithi156.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-09-29T23:19:03.476 INFO:teuthology.orchestra.run.smithi156.stdout:Sourcing file `/etc/default/grub' 2023-09-29T23:19:03.489 INFO:teuthology.orchestra.run.smithi156.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-29T23:19:03.524 INFO:teuthology.orchestra.run.smithi156.stdout:Generating grub configuration file ... 2023-09-29T23:19:03.539 INFO:teuthology.orchestra.run.smithi143.stdout:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-29T23:19:03.560 INFO:teuthology.orchestra.run.smithi143.stdout:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-29T23:19:03.850 INFO:teuthology.orchestra.run.smithi156.stdout:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-29T23:19:03.870 INFO:teuthology.orchestra.run.smithi156.stdout:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-29T23:19:03.920 INFO:teuthology.orchestra.run.smithi143.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-29T23:19:03.931 INFO:teuthology.orchestra.run.smithi143.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-29T23:19:04.266 INFO:teuthology.orchestra.run.smithi156.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-29T23:19:04.278 INFO:teuthology.orchestra.run.smithi156.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-29T23:19:06.394 INFO:teuthology.orchestra.run.smithi143.stdout:done 2023-09-29T23:19:06.653 INFO:teuthology.orchestra.run.smithi156.stdout:done 2023-09-29T23:19:06.794 DEBUG:teuthology.orchestra.run.smithi143:> dpkg -s linux-image-generic 2023-09-29T23:19:06.849 INFO:teuthology.orchestra.run.smithi143.stdout:Package: linux-image-generic 2023-09-29T23:19:06.849 INFO:teuthology.orchestra.run.smithi143.stdout:Status: install ok installed 2023-09-29T23:19:06.850 INFO:teuthology.orchestra.run.smithi143.stdout:Priority: optional 2023-09-29T23:19:06.850 INFO:teuthology.orchestra.run.smithi143.stdout:Section: kernel 2023-09-29T23:19:06.850 INFO:teuthology.orchestra.run.smithi143.stdout:Installed-Size: 20 2023-09-29T23:19:06.850 INFO:teuthology.orchestra.run.smithi143.stdout:Maintainer: Ubuntu Kernel Team 2023-09-29T23:19:06.850 INFO:teuthology.orchestra.run.smithi143.stdout:Architecture: amd64 2023-09-29T23:19:06.850 INFO:teuthology.orchestra.run.smithi143.stdout:Source: linux-meta 2023-09-29T23:19:06.850 INFO:teuthology.orchestra.run.smithi143.stdout:Version: 5.4.0.163.160 2023-09-29T23:19:06.850 INFO:teuthology.orchestra.run.smithi143.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-29T23:19:06.851 INFO:teuthology.orchestra.run.smithi143.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-29T23:19:06.851 INFO:teuthology.orchestra.run.smithi143.stdout:Recommends: thermald 2023-09-29T23:19:06.851 INFO:teuthology.orchestra.run.smithi143.stdout:Description: Generic Linux kernel image 2023-09-29T23:19:06.851 INFO:teuthology.orchestra.run.smithi143.stdout: This package will always depend on the latest generic kernel image 2023-09-29T23:19:06.851 INFO:teuthology.orchestra.run.smithi143.stdout: available. 2023-09-29T23:19:06.851 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-163-generic 2023-09-29T23:19:06.851 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-29T23:19:06.852 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-29T23:19:06.852 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2023-09-29T23:19:06.852 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi143.front.sepia.ceph.com, path=None, version=distro) 2023-09-29T23:19:06.852 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get clean 2023-09-29T23:19:06.940 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get update 2023-09-29T23:19:07.056 DEBUG:teuthology.orchestra.run.smithi156:> dpkg -s linux-image-generic 2023-09-29T23:19:07.082 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-29T23:19:07.082 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-29T23:19:07.092 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-29T23:19:07.112 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-29T23:19:07.115 INFO:teuthology.orchestra.run.smithi156.stdout:Package: linux-image-generic 2023-09-29T23:19:07.115 INFO:teuthology.orchestra.run.smithi156.stdout:Status: install ok installed 2023-09-29T23:19:07.115 INFO:teuthology.orchestra.run.smithi156.stdout:Priority: optional 2023-09-29T23:19:07.115 INFO:teuthology.orchestra.run.smithi156.stdout:Section: kernel 2023-09-29T23:19:07.115 INFO:teuthology.orchestra.run.smithi156.stdout:Installed-Size: 20 2023-09-29T23:19:07.115 INFO:teuthology.orchestra.run.smithi156.stdout:Maintainer: Ubuntu Kernel Team 2023-09-29T23:19:07.115 INFO:teuthology.orchestra.run.smithi156.stdout:Architecture: amd64 2023-09-29T23:19:07.116 INFO:teuthology.orchestra.run.smithi156.stdout:Source: linux-meta 2023-09-29T23:19:07.116 INFO:teuthology.orchestra.run.smithi156.stdout:Version: 5.4.0.163.160 2023-09-29T23:19:07.116 INFO:teuthology.orchestra.run.smithi156.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-29T23:19:07.116 INFO:teuthology.orchestra.run.smithi156.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-29T23:19:07.116 INFO:teuthology.orchestra.run.smithi156.stdout:Recommends: thermald 2023-09-29T23:19:07.116 INFO:teuthology.orchestra.run.smithi156.stdout:Description: Generic Linux kernel image 2023-09-29T23:19:07.116 INFO:teuthology.orchestra.run.smithi156.stdout: This package will always depend on the latest generic kernel image 2023-09-29T23:19:07.116 INFO:teuthology.orchestra.run.smithi156.stdout: available. 2023-09-29T23:19:07.117 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-163-generic 2023-09-29T23:19:07.117 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-29T23:19:07.117 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-29T23:19:07.117 INFO:teuthology.task.kernel:Installing distro kernel on mgr.y... 2023-09-29T23:19:07.117 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi156.front.sepia.ceph.com, path=None, version=distro) 2023-09-29T23:19:07.117 DEBUG:teuthology.orchestra.run.smithi156:> sudo apt-get clean 2023-09-29T23:19:07.202 DEBUG:teuthology.orchestra.run.smithi156:> sudo apt-get update 2023-09-29T23:19:07.331 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-29T23:19:07.337 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-29T23:19:07.354 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-29T23:19:07.449 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:4 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-29T23:19:08.812 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2023-09-29T23:19:08.835 DEBUG:teuthology.orchestra.run.smithi143:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-29T23:19:08.902 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2023-09-29T23:19:09.081 INFO:teuthology.orchestra.run.smithi156.stdout:Reading package lists... 2023-09-29T23:19:09.102 INFO:teuthology.orchestra.run.smithi143.stdout:Building dependency tree... 2023-09-29T23:19:09.103 INFO:teuthology.orchestra.run.smithi143.stdout:Reading state information... 2023-09-29T23:19:09.107 DEBUG:teuthology.orchestra.run.smithi156:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-09-29T23:19:09.173 INFO:teuthology.orchestra.run.smithi156.stdout:Reading package lists... 2023-09-29T23:19:09.317 INFO:teuthology.orchestra.run.smithi143.stdout:linux-image-generic is already the newest version (5.4.0.163.160). 2023-09-29T23:19:09.317 INFO:teuthology.orchestra.run.smithi143.stdout:0 upgraded, 0 newly installed, 0 to remove and 263 not upgraded. 2023-09-29T23:19:09.319 DEBUG:teuthology.orchestra.run.smithi143:> dpkg -s linux-image-generic 2023-09-29T23:19:09.341 INFO:teuthology.orchestra.run.smithi143.stdout:Package: linux-image-generic 2023-09-29T23:19:09.341 INFO:teuthology.orchestra.run.smithi143.stdout:Status: install ok installed 2023-09-29T23:19:09.341 INFO:teuthology.orchestra.run.smithi143.stdout:Priority: optional 2023-09-29T23:19:09.341 INFO:teuthology.orchestra.run.smithi143.stdout:Section: kernel 2023-09-29T23:19:09.342 INFO:teuthology.orchestra.run.smithi143.stdout:Installed-Size: 20 2023-09-29T23:19:09.342 INFO:teuthology.orchestra.run.smithi143.stdout:Maintainer: Ubuntu Kernel Team 2023-09-29T23:19:09.342 INFO:teuthology.orchestra.run.smithi143.stdout:Architecture: amd64 2023-09-29T23:19:09.342 INFO:teuthology.orchestra.run.smithi143.stdout:Source: linux-meta 2023-09-29T23:19:09.342 INFO:teuthology.orchestra.run.smithi143.stdout:Version: 5.4.0.163.160 2023-09-29T23:19:09.342 INFO:teuthology.orchestra.run.smithi143.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-29T23:19:09.342 INFO:teuthology.orchestra.run.smithi143.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-29T23:19:09.342 INFO:teuthology.orchestra.run.smithi143.stdout:Recommends: thermald 2023-09-29T23:19:09.342 INFO:teuthology.orchestra.run.smithi143.stdout:Description: Generic Linux kernel image 2023-09-29T23:19:09.342 INFO:teuthology.orchestra.run.smithi143.stdout: This package will always depend on the latest generic kernel image 2023-09-29T23:19:09.342 INFO:teuthology.orchestra.run.smithi143.stdout: available. 2023-09-29T23:19:09.343 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2023-09-29T23:19:09.348 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.6Z987ol8qj 2023-09-29T23:19:09.348 DEBUG:teuthology.orchestra.run.smithi143:> sudo cp /boot/grub/grub.cfg /tmp/tmp.6Z987ol8qj 2023-09-29T23:19:09.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.6Z987ol8qj 2023-09-29T23:19:09.375 INFO:teuthology.orchestra.run.smithi156.stdout:Building dependency tree... 2023-09-29T23:19:09.375 INFO:teuthology.orchestra.run.smithi156.stdout:Reading state information... 2023-09-29T23:19:09.438 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.6Z987ol8qj is 9KB 2023-09-29T23:19:09.453 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.6Z987ol8qj 2023-09-29T23:19:09.459 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-09-29T23:19:09.518 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:19:09.518 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-09-29T23:19:09.518 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-09-29T23:19:09.563 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-163-generic 2023-09-29T23:19:09.563 DEBUG:teuthology.orchestra.run.smithi143:> sudo update-grub 2023-09-29T23:19:09.582 INFO:teuthology.orchestra.run.smithi156.stdout:linux-image-generic is already the newest version (5.4.0.163.160). 2023-09-29T23:19:09.582 INFO:teuthology.orchestra.run.smithi156.stdout:0 upgraded, 0 newly installed, 0 to remove and 263 not upgraded. 2023-09-29T23:19:09.584 DEBUG:teuthology.orchestra.run.smithi156:> dpkg -s linux-image-generic 2023-09-29T23:19:09.599 INFO:teuthology.orchestra.run.smithi156.stdout:Package: linux-image-generic 2023-09-29T23:19:09.599 INFO:teuthology.orchestra.run.smithi156.stdout:Status: install ok installed 2023-09-29T23:19:09.599 INFO:teuthology.orchestra.run.smithi156.stdout:Priority: optional 2023-09-29T23:19:09.599 INFO:teuthology.orchestra.run.smithi156.stdout:Section: kernel 2023-09-29T23:19:09.599 INFO:teuthology.orchestra.run.smithi156.stdout:Installed-Size: 20 2023-09-29T23:19:09.599 INFO:teuthology.orchestra.run.smithi156.stdout:Maintainer: Ubuntu Kernel Team 2023-09-29T23:19:09.600 INFO:teuthology.orchestra.run.smithi156.stdout:Architecture: amd64 2023-09-29T23:19:09.600 INFO:teuthology.orchestra.run.smithi156.stdout:Source: linux-meta 2023-09-29T23:19:09.600 INFO:teuthology.orchestra.run.smithi156.stdout:Version: 5.4.0.163.160 2023-09-29T23:19:09.600 INFO:teuthology.orchestra.run.smithi156.stdout:Provides: virtualbox-guest-modules (= 6.1.38-dfsg-3~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.15) 2023-09-29T23:19:09.600 INFO:teuthology.orchestra.run.smithi156.stdout:Depends: linux-image-5.4.0-163-generic, linux-modules-extra-5.4.0-163-generic, linux-firmware, intel-microcode, amd64-microcode 2023-09-29T23:19:09.600 INFO:teuthology.orchestra.run.smithi156.stdout:Recommends: thermald 2023-09-29T23:19:09.600 INFO:teuthology.orchestra.run.smithi156.stdout:Description: Generic Linux kernel image 2023-09-29T23:19:09.600 INFO:teuthology.orchestra.run.smithi156.stdout: This package will always depend on the latest generic kernel image 2023-09-29T23:19:09.600 INFO:teuthology.orchestra.run.smithi156.stdout: available. 2023-09-29T23:19:09.600 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2023-09-29T23:19:09.606 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.CPfjdjyvvo 2023-09-29T23:19:09.606 DEBUG:teuthology.orchestra.run.smithi156:> sudo cp /boot/grub/grub.cfg /tmp/tmp.CPfjdjyvvo 2023-09-29T23:19:09.617 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.CPfjdjyvvo 2023-09-29T23:19:09.706 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.CPfjdjyvvo is 9KB 2023-09-29T23:19:09.719 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.CPfjdjyvvo 2023-09-29T23:19:09.725 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-09-29T23:19:09.737 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:19:09.738 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-09-29T23:19:09.738 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-09-29T23:19:09.765 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-163-generic 2023-09-29T23:19:09.765 DEBUG:teuthology.orchestra.run.smithi156:> sudo update-grub 2023-09-29T23:19:10.846 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub' 2023-09-29T23:19:10.855 INFO:teuthology.orchestra.run.smithi143.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-29T23:19:10.864 INFO:teuthology.orchestra.run.smithi143.stderr:Generating grub configuration file ... 2023-09-29T23:19:11.069 INFO:teuthology.orchestra.run.smithi156.stderr:Sourcing file `/etc/default/grub' 2023-09-29T23:19:11.077 INFO:teuthology.orchestra.run.smithi156.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-09-29T23:19:11.089 INFO:teuthology.orchestra.run.smithi156.stderr:Generating grub configuration file ... 2023-09-29T23:19:11.105 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-29T23:19:11.117 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-29T23:19:11.327 INFO:teuthology.orchestra.run.smithi156.stderr:Found linux image: /boot/vmlinuz-5.4.0-163-generic 2023-09-29T23:19:11.339 INFO:teuthology.orchestra.run.smithi156.stderr:Found initrd image: /boot/initrd.img-5.4.0-163-generic 2023-09-29T23:19:11.469 INFO:teuthology.orchestra.run.smithi143.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-29T23:19:11.479 INFO:teuthology.orchestra.run.smithi143.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-29T23:19:11.695 INFO:teuthology.orchestra.run.smithi156.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2023-09-29T23:19:11.707 INFO:teuthology.orchestra.run.smithi156.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2023-09-29T23:19:12.875 INFO:teuthology.orchestra.run.smithi143.stderr:done 2023-09-29T23:19:12.877 DEBUG:teuthology.orchestra.run.smithi143:> sudo shutdown -r now 2023-09-29T23:19:13.161 INFO:teuthology.orchestra.run.smithi156.stderr:done 2023-09-29T23:19:13.163 DEBUG:teuthology.orchestra.run.smithi156:> sudo shutdown -r now 2023-09-29T23:19:42.908 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2023-09-29T23:19:42.908 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T23:19:42.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:19:43.166 INFO:teuthology.task.kernel:Checking client mgr.y for new kernel version... 2023-09-29T23:19:43.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-09-29T23:19:43.166 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:20:01.407 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2023-09-29T23:20:01.662 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-09-29T23:20:10.415 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T23:20:10.416 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:20:10.664 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-09-29T23:20:10.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:20:13.466 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2023-09-29T23:20:13.726 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-09-29T23:20:25.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T23:20:25.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:20:25.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-09-29T23:20:25.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:20:43.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.143 2023-09-29T23:20:44.154 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-09-29T23:20:58.918 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T23:20:58.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:20:59.135 DEBUG:teuthology.orchestra.run.smithi143:> true 2023-09-29T23:20:59.155 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-09-29T23:20:59.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:20:59.537 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-09-29T23:20:59.726 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T23:20:59.727 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.4.0-163-generic"... 2023-09-29T23:20:59.727 DEBUG:teuthology.orchestra.run.smithi143:> uname -r 2023-09-29T23:20:59.733 INFO:teuthology.orchestra.run.smithi143.stdout:5.4.0-163-generic 2023-09-29T23:20:59.734 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-163-generic vs 5.4.0-163-generic 2023-09-29T23:20:59.734 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-29T23:20:59.734 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-09-29T23:21:00.470 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-09-29T23:21:00.470 INFO:teuthology.task.kernel:Checking kernel version of mgr.y, want "5.4.0-163-generic"... 2023-09-29T23:21:00.470 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2023-09-29T23:21:00.476 INFO:teuthology.orchestra.run.smithi156.stdout:5.4.0-163-generic 2023-09-29T23:21:00.477 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-163-generic vs 5.4.0-163-generic 2023-09-29T23:21:00.477 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-29T23:21:00.477 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-09-29T23:21:00.735 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2023-09-29T23:21:00.735 DEBUG:teuthology.orchestra.run.smithi143:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-29T23:21:00.815 INFO:teuthology.orchestra.run.smithi143.stdout:ttyS1 2023-09-29T23:21:00.840 DEBUG:teuthology.parallel:result is None 2023-09-29T23:21:01.478 INFO:teuthology.task.kernel:Enabling kdb on mgr.y... 2023-09-29T23:21:01.478 DEBUG:teuthology.orchestra.run.smithi156:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-29T23:21:01.945 INFO:teuthology.orchestra.run.smithi156.stdout:ttyS1 2023-09-29T23:21:01.993 DEBUG:teuthology.parallel:result is None 2023-09-29T23:21:01.993 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-29T23:21:02.000 INFO:teuthology.task.internal:Creating test directory... 2023-09-29T23:21:02.000 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-29T23:21:02.003 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-29T23:21:02.009 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-29T23:21:02.018 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-29T23:21:02.029 INFO:teuthology.task.internal:Creating archive directory... 2023-09-29T23:21:02.029 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-29T23:21:02.052 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-29T23:21:02.080 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-29T23:21:02.088 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-29T23:21:02.088 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-29T23:21:02.090 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-29T23:21:02.106 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-29T23:21:02.117 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-29T23:21:02.143 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-29T23:21:02.152 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-29T23:21:02.153 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-29T23:21:02.161 INFO:teuthology.task.internal:Configuring sudo... 2023-09-29T23:21:02.162 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-29T23:21:02.164 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-29T23:21:02.178 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-29T23:21:02.186 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-29T23:21:02.186 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-29T23:21:02.189 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-29T23:21:02.224 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-29T23:21:02.231 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-29T23:21:02.280 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:21:02.280 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-29T23:21:02.308 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-29T23:21:02.316 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-29T23:21:02.322 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:21:02.323 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-29T23:21:02.352 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2023-09-29T23:21:02.354 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2023-09-29T23:21:02.454 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-29T23:21:02.462 INFO:teuthology.task.internal:Starting timer... 2023-09-29T23:21:02.462 INFO:teuthology.run_tasks:Running task pcp... 2023-09-29T23:21:02.474 INFO:teuthology.run_tasks:Running task selinux... 2023-09-29T23:21:02.484 DEBUG:teuthology.task.selinux:Excluding smithi143: OS 'ubuntu' does not support SELinux 2023-09-29T23:21:02.484 DEBUG:teuthology.task.selinux:Excluding smithi156: OS 'ubuntu' does not support SELinux 2023-09-29T23:21:02.484 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-29T23:21:02.484 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-09-29T23:21:02.484 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-29T23:21:02.485 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-29T23:21:02.494 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-09-29T23:21:02.494 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-29T23:21:02.510 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-29T23:21:02.511 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi143.front.sepia.ceph.com,smithi156.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-29T23:26:46.136 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi143.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com')] 2023-09-29T23:26:46.137 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T23:26:46.138 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:26:46.186 DEBUG:teuthology.orchestra.run.smithi143:> true 2023-09-29T23:26:46.307 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2023-09-29T23:26:46.308 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-09-29T23:26:46.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-29T23:26:46.354 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-09-29T23:26:46.476 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-09-29T23:26:46.476 INFO:teuthology.run_tasks:Running task clock... 2023-09-29T23:26:46.486 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-29T23:26:46.486 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-29T23:26:46.486 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-29T23:26:46.488 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-29T23:26:46.489 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-29T23:26:46.518 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-09-29T23:26:46.518 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: Command line: ntpd -gq 2023-09-29T23:26:46.519 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: proto: precision = 0.055 usec (-24) 2023-09-29T23:26:46.519 INFO:teuthology.orchestra.run.smithi143.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-29T23:26:46.520 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-29T23:26:46.520 INFO:teuthology.orchestra.run.smithi143.stderr:restrict ::: KOD does nothing without LIMITED. 2023-09-29T23:26:46.521 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: restrict ::: KOD does nothing without LIMITED. 2023-09-29T23:26:46.521 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: Listen and drop on 0 v6wildcard [::]:123 2023-09-29T23:26:46.521 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-09-29T23:26:46.521 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: Listen normally on 2 lo 127.0.0.1:123 2023-09-29T23:26:46.521 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: Listen normally on 3 enp3s0f1 172.21.15.143:123 2023-09-29T23:26:46.521 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: Listen normally on 4 lo [::1]:123 2023-09-29T23:26:46.521 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7783%5]:123 2023-09-29T23:26:46.521 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:46 ntpd[22710]: Listening on routing socket on fd #22 for interface updates 2023-09-29T23:26:46.524 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: ntpd 4.2.8p12@1.3728-o (1): Starting 2023-09-29T23:26:46.525 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: Command line: ntpd -gq 2023-09-29T23:26:46.526 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: proto: precision = 0.190 usec (-22) 2023-09-29T23:26:46.526 INFO:teuthology.orchestra.run.smithi156.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-29T23:26:46.526 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-09-29T23:26:46.527 INFO:teuthology.orchestra.run.smithi156.stderr:restrict ::: KOD does nothing without LIMITED. 2023-09-29T23:26:46.528 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: restrict ::: KOD does nothing without LIMITED. 2023-09-29T23:26:46.528 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: Listen and drop on 0 v6wildcard [::]:123 2023-09-29T23:26:46.528 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-09-29T23:26:46.528 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: Listen normally on 2 lo 127.0.0.1:123 2023-09-29T23:26:46.528 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: Listen normally on 3 enp3s0f1 172.21.15.156:123 2023-09-29T23:26:46.528 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: Listen normally on 4 lo [::1]:123 2023-09-29T23:26:46.528 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6dfd%5]:123 2023-09-29T23:26:46.528 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:46 ntpd[22700]: Listening on routing socket on fd #22 for interface updates 2023-09-29T23:26:47.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:47 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:47.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:47 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:47.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:47 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:47.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:47 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:47.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:47 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:47.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:47 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:47.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:47 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:47.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:47 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:48.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:48 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:48.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:48 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:48.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:48 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:48.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:48 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:48.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:48 ntpd[22700]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:48.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:48 ntpd[22700]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:49.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:49 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:49.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:49 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:49.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:49 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:49.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:49 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:49.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:49 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:49.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:49 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:49.525 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:49 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:49.525 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:49 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:50.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:50.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:50.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:50.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:50.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:50.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:50.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:50.521 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:50.521 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:50.521 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:50.521 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:50.521 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:50 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:50.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:50 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:50.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:50 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:50.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:50 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:50.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:50 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:50.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:50 ntpd[22700]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:50.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:50 ntpd[22700]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:51 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:51 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:51 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:51 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:51 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:51.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:51 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:51.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:51.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:51.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:51.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:51.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:51.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:51.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:51.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:51.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:51.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:51.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:51.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:51 ntpd[22700]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:52.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:52.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:52.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:52.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:52.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:52.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:52.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:52.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:52.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:52.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:52.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:52.521 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:52 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:52.525 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:52 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:52.525 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:52 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:52 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:52 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:52 ntpd[22700]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:52.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:52 ntpd[22700]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:53.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:53 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:53.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:53 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:53.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:53 ntpd[22710]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:53.519 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:53 ntpd[22710]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:53.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:53 ntpd[22710]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:53.520 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:53 ntpd[22710]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:53.520 INFO:teuthology.orchestra.run.smithi143.stdout:29 Sep 23:26:53 ntpd[22710]: ntpd: time slew -0.000684 s 2023-09-29T23:26:53.520 INFO:teuthology.orchestra.run.smithi143.stdout:ntpd: time slew -0.000684s 2023-09-29T23:26:53.521 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:53 ntpd[22710]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-09-29T23:26:53.521 INFO:teuthology.orchestra.run.smithi143.stderr:29 Sep 23:26:53 ntpd[22710]: can't open /var/log/ntpstats/loopstats.20230929: Permission denied 2023-09-29T23:26:53.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:53.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:53.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:53.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:53.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:53.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:53.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:53.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:53.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:53.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:53.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:53.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:53 ntpd[22700]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:53.591 INFO:teuthology.orchestra.run.smithi143.stdout: remote refid st t when poll reach delay offset jitter 2023-09-29T23:26:53.591 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================== 2023-09-29T23:26:53.591 INFO:teuthology.orchestra.run.smithi143.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-29T23:26:53.591 INFO:teuthology.orchestra.run.smithi143.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-29T23:26:53.591 INFO:teuthology.orchestra.run.smithi143.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-29T23:26:53.591 INFO:teuthology.orchestra.run.smithi143.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-29T23:26:54.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:54 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:54.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:54 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:54.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:54 ntpd[22700]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-09-29T23:26:54.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:54 ntpd[22700]: can't open /var/log/ntpstats/rawstats.20230929: Permission denied 2023-09-29T23:26:54.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:54 ntpd[22700]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-09-29T23:26:54.526 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:54 ntpd[22700]: can't open /var/log/ntpstats/peerstats.20230929: Permission denied 2023-09-29T23:26:54.527 INFO:teuthology.orchestra.run.smithi156.stdout:29 Sep 23:26:54 ntpd[22700]: ntpd: time slew +0.001598 s 2023-09-29T23:26:54.527 INFO:teuthology.orchestra.run.smithi156.stdout:ntpd: time slew +0.001598s 2023-09-29T23:26:54.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:54 ntpd[22700]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-09-29T23:26:54.527 INFO:teuthology.orchestra.run.smithi156.stderr:29 Sep 23:26:54 ntpd[22700]: can't open /var/log/ntpstats/loopstats.20230929: Permission denied 2023-09-29T23:26:54.587 INFO:teuthology.orchestra.run.smithi156.stdout: remote refid st t when poll reach delay offset jitter 2023-09-29T23:26:54.588 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================== 2023-09-29T23:26:54.588 INFO:teuthology.orchestra.run.smithi156.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-29T23:26:54.588 INFO:teuthology.orchestra.run.smithi156.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-29T23:26:54.588 INFO:teuthology.orchestra.run.smithi156.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-29T23:26:54.588 INFO:teuthology.orchestra.run.smithi156.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-29T23:26:54.589 INFO:teuthology.run_tasks:Running task install... 2023-09-29T23:26:54.598 DEBUG:teuthology.task.install:project ceph 2023-09-29T23:26:54.598 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}} 2023-09-29T23:26:54.598 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'} 2023-09-29T23:26:54.598 INFO:teuthology.task.install:Using flavor: default 2023-09-29T23:26:54.604 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-29T23:26:54.605 INFO:teuthology.task.install:extra packages: [] 2023-09-29T23:26:54.605 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-key list | grep Ceph 2023-09-29T23:26:54.605 DEBUG:teuthology.orchestra.run.smithi156:> sudo apt-key list | grep Ceph 2023-09-29T23:26:54.705 INFO:teuthology.orchestra.run.smithi143.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-09-29T23:26:54.708 INFO:teuthology.orchestra.run.smithi156.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2023-09-29T23:26:54.765 INFO:teuthology.orchestra.run.smithi143.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-09-29T23:26:54.765 INFO:teuthology.orchestra.run.smithi143.stdout:uid [ unknown] Ceph.com (release key) 2023-09-29T23:26:54.766 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-09-29T23:26:54.767 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-29T23:26:54.771 INFO:teuthology.orchestra.run.smithi156.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-09-29T23:26:54.771 INFO:teuthology.orchestra.run.smithi156.stdout:uid [ unknown] Ceph.com (release key) 2023-09-29T23:26:54.771 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2023-09-29T23:26:54.771 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-29T23:26:54.914 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default/ 2023-09-29T23:26:54.914 INFO:teuthology.task.install.deb:Package version is 17.2.6-1165-g93dce5e1-1focal 2023-09-29T23:26:54.986 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default/ 2023-09-29T23:26:54.986 INFO:teuthology.task.install.deb:Package version is 17.2.6-1165-g93dce5e1-1focal 2023-09-29T23:26:55.093 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:26:55.093 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-09-29T23:26:55.095 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:26:55.095 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-09-29T23:26:55.111 DEBUG:teuthology.orchestra.run.smithi156:> sudo apt-get update 2023-09-29T23:26:55.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo apt-get update 2023-09-29T23:26:55.254 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-29T23:26:55.262 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-29T23:26:55.263 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2023-09-29T23:26:55.272 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2023-09-29T23:26:55.282 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-29T23:26:55.292 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2023-09-29T23:26:55.309 INFO:teuthology.orchestra.run.smithi156.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal InRelease 2023-09-29T23:26:55.330 INFO:teuthology.orchestra.run.smithi156.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-09-29T23:26:55.353 INFO:teuthology.orchestra.run.smithi156.stdout:Ign:6 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release.gpg 2023-09-29T23:26:55.364 INFO:teuthology.orchestra.run.smithi143.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal InRelease 2023-09-29T23:26:55.367 INFO:teuthology.orchestra.run.smithi143.stdout:Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-29T23:26:55.372 INFO:teuthology.orchestra.run.smithi156.stdout:Hit:7 http://security.ubuntu.com/ubuntu focal-security InRelease 2023-09-29T23:26:55.375 INFO:teuthology.orchestra.run.smithi156.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-09-29T23:26:55.401 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release [7,662 B] 2023-09-29T23:26:55.410 INFO:teuthology.orchestra.run.smithi156.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main i386 Packages [3,375 B] 2023-09-29T23:26:55.450 INFO:teuthology.orchestra.run.smithi143.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal Release.gpg 2023-09-29T23:26:55.486 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2023-09-29T23:26:55.543 INFO:teuthology.orchestra.run.smithi143.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main i386 Packages [3,375 B] 2023-09-29T23:26:55.887 INFO:teuthology.orchestra.run.smithi156.stdout:Fetched 28.4 kB in 1s (45.7 kB/s) 2023-09-29T23:26:55.909 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 28.4 kB in 1s (44.5 kB/s) 2023-09-29T23:26:56.926 INFO:teuthology.orchestra.run.smithi156.stdout:Reading package lists... 2023-09-29T23:26:56.942 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2023-09-29T23:26:56.952 DEBUG:teuthology.orchestra.run.smithi156:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1165-g93dce5e1-1focal cephadm=17.2.6-1165-g93dce5e1-1focal ceph-mds=17.2.6-1165-g93dce5e1-1focal ceph-mgr=17.2.6-1165-g93dce5e1-1focal ceph-common=17.2.6-1165-g93dce5e1-1focal ceph-fuse=17.2.6-1165-g93dce5e1-1focal ceph-test=17.2.6-1165-g93dce5e1-1focal ceph-volume=17.2.6-1165-g93dce5e1-1focal radosgw=17.2.6-1165-g93dce5e1-1focal python3-rados=17.2.6-1165-g93dce5e1-1focal python3-rgw=17.2.6-1165-g93dce5e1-1focal python3-cephfs=17.2.6-1165-g93dce5e1-1focal python3-rbd=17.2.6-1165-g93dce5e1-1focal libcephfs2=17.2.6-1165-g93dce5e1-1focal libcephfs-dev=17.2.6-1165-g93dce5e1-1focal librados2=17.2.6-1165-g93dce5e1-1focal librbd1=17.2.6-1165-g93dce5e1-1focal rbd-fuse=17.2.6-1165-g93dce5e1-1focal 2023-09-29T23:26:56.968 DEBUG:teuthology.orchestra.run.smithi143:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.6-1165-g93dce5e1-1focal cephadm=17.2.6-1165-g93dce5e1-1focal ceph-mds=17.2.6-1165-g93dce5e1-1focal ceph-mgr=17.2.6-1165-g93dce5e1-1focal ceph-common=17.2.6-1165-g93dce5e1-1focal ceph-fuse=17.2.6-1165-g93dce5e1-1focal ceph-test=17.2.6-1165-g93dce5e1-1focal ceph-volume=17.2.6-1165-g93dce5e1-1focal radosgw=17.2.6-1165-g93dce5e1-1focal python3-rados=17.2.6-1165-g93dce5e1-1focal python3-rgw=17.2.6-1165-g93dce5e1-1focal python3-cephfs=17.2.6-1165-g93dce5e1-1focal python3-rbd=17.2.6-1165-g93dce5e1-1focal libcephfs2=17.2.6-1165-g93dce5e1-1focal libcephfs-dev=17.2.6-1165-g93dce5e1-1focal librados2=17.2.6-1165-g93dce5e1-1focal librbd1=17.2.6-1165-g93dce5e1-1focal rbd-fuse=17.2.6-1165-g93dce5e1-1focal 2023-09-29T23:26:57.022 INFO:teuthology.orchestra.run.smithi156.stdout:Reading package lists... 2023-09-29T23:26:57.035 INFO:teuthology.orchestra.run.smithi143.stdout:Reading package lists... 2023-09-29T23:26:57.201 INFO:teuthology.orchestra.run.smithi156.stdout:Building dependency tree... 2023-09-29T23:26:57.201 INFO:teuthology.orchestra.run.smithi156.stdout:Reading state information... 2023-09-29T23:26:57.236 INFO:teuthology.orchestra.run.smithi143.stdout:Building dependency tree... 2023-09-29T23:26:57.236 INFO:teuthology.orchestra.run.smithi143.stdout:Reading state information... 2023-09-29T23:26:57.372 INFO:teuthology.orchestra.run.smithi156.stdout:The following packages were automatically installed and are no longer required: 2023-09-29T23:26:57.373 INFO:teuthology.orchestra.run.smithi156.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-09-29T23:26:57.374 INFO:teuthology.orchestra.run.smithi156.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-29T23:26:57.374 INFO:teuthology.orchestra.run.smithi156.stdout:The following additional packages will be installed: 2023-09-29T23:26:57.374 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-09-29T23:26:57.374 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-09-29T23:26:57.374 INFO:teuthology.orchestra.run.smithi156.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-09-29T23:26:57.374 INFO:teuthology.orchestra.run.smithi156.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-09-29T23:26:57.374 INFO:teuthology.orchestra.run.smithi156.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-09-29T23:26:57.375 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-09-29T23:26:57.375 INFO:teuthology.orchestra.run.smithi156.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-09-29T23:26:57.375 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-09-29T23:26:57.375 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2023-09-29T23:26:57.376 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-09-29T23:26:57.376 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-09-29T23:26:57.376 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-09-29T23:26:57.376 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-09-29T23:26:57.376 INFO:teuthology.orchestra.run.smithi156.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-09-29T23:26:57.376 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-09-29T23:26:57.376 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2023-09-29T23:26:57.376 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2023-09-29T23:26:57.376 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-09-29T23:26:57.376 INFO:teuthology.orchestra.run.smithi156.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2023-09-29T23:26:57.377 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2023-09-29T23:26:57.377 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2023-09-29T23:26:57.377 INFO:teuthology.orchestra.run.smithi156.stdout: socat xmlstarlet 2023-09-29T23:26:57.377 INFO:teuthology.orchestra.run.smithi156.stdout:Suggested packages: 2023-09-29T23:26:57.377 INFO:teuthology.orchestra.run.smithi156.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-09-29T23:26:57.377 INFO:teuthology.orchestra.run.smithi156.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-09-29T23:26:57.377 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-09-29T23:26:57.378 INFO:teuthology.orchestra.run.smithi156.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-09-29T23:26:57.378 INFO:teuthology.orchestra.run.smithi156.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-09-29T23:26:57.378 INFO:teuthology.orchestra.run.smithi156.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-09-29T23:26:57.378 INFO:teuthology.orchestra.run.smithi156.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-09-29T23:26:57.379 INFO:teuthology.orchestra.run.smithi156.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-09-29T23:26:57.379 INFO:teuthology.orchestra.run.smithi156.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-09-29T23:26:57.379 INFO:teuthology.orchestra.run.smithi156.stdout:Recommended packages: 2023-09-29T23:26:57.379 INFO:teuthology.orchestra.run.smithi156.stdout: btrfs-tools 2023-09-29T23:26:57.407 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages were automatically installed and are no longer required: 2023-09-29T23:26:57.408 INFO:teuthology.orchestra.run.smithi143.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2023-09-29T23:26:57.408 INFO:teuthology.orchestra.run.smithi143.stdout:Use 'sudo apt autoremove' to remove them. 2023-09-29T23:26:57.408 INFO:teuthology.orchestra.run.smithi143.stdout:The following additional packages will be installed: 2023-09-29T23:26:57.409 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-09-29T23:26:57.409 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2023-09-29T23:26:57.409 INFO:teuthology.orchestra.run.smithi143.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2023-09-29T23:26:57.409 INFO:teuthology.orchestra.run.smithi143.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2023-09-29T23:26:57.409 INFO:teuthology.orchestra.run.smithi143.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2023-09-29T23:26:57.411 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2023-09-29T23:26:57.411 INFO:teuthology.orchestra.run.smithi143.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2023-09-29T23:26:57.411 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2023-09-29T23:26:57.411 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2023-09-29T23:26:57.411 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2023-09-29T23:26:57.411 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2023-09-29T23:26:57.411 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2023-09-29T23:26:57.411 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2023-09-29T23:26:57.411 INFO:teuthology.orchestra.run.smithi143.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout: socat xmlstarlet 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout:Suggested packages: 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2023-09-29T23:26:57.412 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2023-09-29T23:26:57.413 INFO:teuthology.orchestra.run.smithi143.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2023-09-29T23:26:57.413 INFO:teuthology.orchestra.run.smithi143.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2023-09-29T23:26:57.413 INFO:teuthology.orchestra.run.smithi143.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2023-09-29T23:26:57.413 INFO:teuthology.orchestra.run.smithi143.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2023-09-29T23:26:57.413 INFO:teuthology.orchestra.run.smithi143.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-09-29T23:26:57.413 INFO:teuthology.orchestra.run.smithi143.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2023-09-29T23:26:57.413 INFO:teuthology.orchestra.run.smithi143.stdout:Recommended packages: 2023-09-29T23:26:57.413 INFO:teuthology.orchestra.run.smithi143.stdout: btrfs-tools 2023-09-29T23:26:57.491 INFO:teuthology.orchestra.run.smithi156.stdout:The following NEW packages will be installed: 2023-09-29T23:26:57.492 INFO:teuthology.orchestra.run.smithi156.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-09-29T23:26:57.492 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-09-29T23:26:57.492 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-09-29T23:26:57.492 INFO:teuthology.orchestra.run.smithi156.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2023-09-29T23:26:57.492 INFO:teuthology.orchestra.run.smithi156.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2023-09-29T23:26:57.492 INFO:teuthology.orchestra.run.smithi156.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2023-09-29T23:26:57.492 INFO:teuthology.orchestra.run.smithi156.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2023-09-29T23:26:57.493 INFO:teuthology.orchestra.run.smithi156.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2023-09-29T23:26:57.493 INFO:teuthology.orchestra.run.smithi156.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2023-09-29T23:26:57.493 INFO:teuthology.orchestra.run.smithi156.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-09-29T23:26:57.493 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2023-09-29T23:26:57.493 INFO:teuthology.orchestra.run.smithi156.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-09-29T23:26:57.493 INFO:teuthology.orchestra.run.smithi156.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-09-29T23:26:57.493 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2023-09-29T23:26:57.493 INFO:teuthology.orchestra.run.smithi156.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2023-09-29T23:26:57.493 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2023-09-29T23:26:57.493 INFO:teuthology.orchestra.run.smithi156.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-09-29T23:26:57.494 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2023-09-29T23:26:57.494 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2023-09-29T23:26:57.494 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-09-29T23:26:57.494 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-09-29T23:26:57.494 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-09-29T23:26:57.494 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-09-29T23:26:57.494 INFO:teuthology.orchestra.run.smithi156.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-09-29T23:26:57.495 INFO:teuthology.orchestra.run.smithi156.stdout:The following packages will be upgraded: 2023-09-29T23:26:57.495 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 librbd1 2023-09-29T23:26:57.528 INFO:teuthology.orchestra.run.smithi143.stdout:The following NEW packages will be installed: 2023-09-29T23:26:57.528 INFO:teuthology.orchestra.run.smithi143.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-09-29T23:26:57.528 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-09-29T23:26:57.529 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-09-29T23:26:57.531 INFO:teuthology.orchestra.run.smithi143.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2023-09-29T23:26:57.531 INFO:teuthology.orchestra.run.smithi143.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2023-09-29T23:26:57.531 INFO:teuthology.orchestra.run.smithi143.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2023-09-29T23:26:57.531 INFO:teuthology.orchestra.run.smithi143.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2023-09-29T23:26:57.531 INFO:teuthology.orchestra.run.smithi143.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2023-09-29T23:26:57.531 INFO:teuthology.orchestra.run.smithi143.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2023-09-29T23:26:57.531 INFO:teuthology.orchestra.run.smithi143.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2023-09-29T23:26:57.531 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2023-09-29T23:26:57.531 INFO:teuthology.orchestra.run.smithi143.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2023-09-29T23:26:57.531 INFO:teuthology.orchestra.run.smithi143.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2023-09-29T23:26:57.531 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2023-09-29T23:26:57.532 INFO:teuthology.orchestra.run.smithi143.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2023-09-29T23:26:57.532 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2023-09-29T23:26:57.532 INFO:teuthology.orchestra.run.smithi143.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-09-29T23:26:57.532 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2023-09-29T23:26:57.532 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2023-09-29T23:26:57.532 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2023-09-29T23:26:57.532 INFO:teuthology.orchestra.run.smithi143.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2023-09-29T23:26:57.532 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2023-09-29T23:26:57.532 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2023-09-29T23:26:57.532 INFO:teuthology.orchestra.run.smithi143.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2023-09-29T23:26:57.532 INFO:teuthology.orchestra.run.smithi143.stdout:The following packages will be upgraded: 2023-09-29T23:26:57.533 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 librbd1 2023-09-29T23:26:57.562 INFO:teuthology.orchestra.run.smithi156.stdout:2 upgraded, 108 newly installed, 0 to remove and 245 not upgraded. 2023-09-29T23:26:57.563 INFO:teuthology.orchestra.run.smithi156.stdout:Need to get 209 MB of archives. 2023-09-29T23:26:57.563 INFO:teuthology.orchestra.run.smithi156.stdout:After this operation, 980 MB of additional disk space will be used. 2023-09-29T23:26:57.563 INFO:teuthology.orchestra.run.smithi156.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-09-29T23:26:57.596 INFO:teuthology.orchestra.run.smithi143.stdout:2 upgraded, 108 newly installed, 0 to remove and 245 not upgraded. 2023-09-29T23:26:57.596 INFO:teuthology.orchestra.run.smithi143.stdout:Need to get 209 MB of archives. 2023-09-29T23:26:57.596 INFO:teuthology.orchestra.run.smithi143.stdout:After this operation, 980 MB of additional disk space will be used. 2023-09-29T23:26:57.596 INFO:teuthology.orchestra.run.smithi143.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2023-09-29T23:26:57.644 INFO:teuthology.orchestra.run.smithi156.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1165-g93dce5e1-1focal [3,501 kB] 2023-09-29T23:26:57.658 INFO:teuthology.orchestra.run.smithi156.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-09-29T23:26:57.660 INFO:teuthology.orchestra.run.smithi156.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-09-29T23:26:57.670 INFO:teuthology.orchestra.run.smithi156.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-09-29T23:26:57.678 INFO:teuthology.orchestra.run.smithi156.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-09-29T23:26:57.678 INFO:teuthology.orchestra.run.smithi143.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.6-1165-g93dce5e1-1focal [3,501 kB] 2023-09-29T23:26:57.693 INFO:teuthology.orchestra.run.smithi143.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2023-09-29T23:26:57.696 INFO:teuthology.orchestra.run.smithi143.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2023-09-29T23:26:57.706 INFO:teuthology.orchestra.run.smithi156.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-09-29T23:26:57.706 INFO:teuthology.orchestra.run.smithi143.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2023-09-29T23:26:57.707 INFO:teuthology.orchestra.run.smithi156.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-09-29T23:26:57.715 INFO:teuthology.orchestra.run.smithi143.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2023-09-29T23:26:57.716 INFO:teuthology.orchestra.run.smithi156.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-09-29T23:26:57.744 INFO:teuthology.orchestra.run.smithi143.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2023-09-29T23:26:57.745 INFO:teuthology.orchestra.run.smithi143.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2023-09-29T23:26:57.756 INFO:teuthology.orchestra.run.smithi156.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-09-29T23:26:57.756 INFO:teuthology.orchestra.run.smithi143.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2023-09-29T23:26:57.758 INFO:teuthology.orchestra.run.smithi156.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-09-29T23:26:57.768 INFO:teuthology.orchestra.run.smithi156.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-09-29T23:26:57.772 INFO:teuthology.orchestra.run.smithi156.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-09-29T23:26:57.773 INFO:teuthology.orchestra.run.smithi156.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-09-29T23:26:57.776 INFO:teuthology.orchestra.run.smithi156.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-09-29T23:26:57.777 INFO:teuthology.orchestra.run.smithi156.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-09-29T23:26:57.777 INFO:teuthology.orchestra.run.smithi156.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-09-29T23:26:57.781 INFO:teuthology.orchestra.run.smithi156.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-09-29T23:26:57.788 INFO:teuthology.orchestra.run.smithi156.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-09-29T23:26:57.801 INFO:teuthology.orchestra.run.smithi156.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-09-29T23:26:57.802 INFO:teuthology.orchestra.run.smithi156.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-09-29T23:26:57.802 INFO:teuthology.orchestra.run.smithi156.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-09-29T23:26:57.803 INFO:teuthology.orchestra.run.smithi156.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-09-29T23:26:57.803 INFO:teuthology.orchestra.run.smithi156.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-09-29T23:26:57.804 INFO:teuthology.orchestra.run.smithi156.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-09-29T23:26:57.805 INFO:teuthology.orchestra.run.smithi156.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-09-29T23:26:57.806 INFO:teuthology.orchestra.run.smithi156.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-09-29T23:26:57.811 INFO:teuthology.orchestra.run.smithi143.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2023-09-29T23:26:57.812 INFO:teuthology.orchestra.run.smithi156.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-09-29T23:26:57.815 INFO:teuthology.orchestra.run.smithi143.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2023-09-29T23:26:57.823 INFO:teuthology.orchestra.run.smithi143.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2023-09-29T23:26:57.826 INFO:teuthology.orchestra.run.smithi143.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2023-09-29T23:26:57.826 INFO:teuthology.orchestra.run.smithi143.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.1 [211 kB] 2023-09-29T23:26:57.829 INFO:teuthology.orchestra.run.smithi143.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2023-09-29T23:26:57.829 INFO:teuthology.orchestra.run.smithi143.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2023-09-29T23:26:57.830 INFO:teuthology.orchestra.run.smithi143.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2023-09-29T23:26:57.831 INFO:teuthology.orchestra.run.smithi143.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2023-09-29T23:26:57.832 INFO:teuthology.orchestra.run.smithi156.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-09-29T23:26:57.832 INFO:teuthology.orchestra.run.smithi156.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-09-29T23:26:57.838 INFO:teuthology.orchestra.run.smithi156.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-09-29T23:26:57.839 INFO:teuthology.orchestra.run.smithi156.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-09-29T23:26:57.839 INFO:teuthology.orchestra.run.smithi143.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2023-09-29T23:26:57.840 INFO:teuthology.orchestra.run.smithi156.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-09-29T23:26:57.841 INFO:teuthology.orchestra.run.smithi156.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-09-29T23:26:57.842 INFO:teuthology.orchestra.run.smithi156.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-09-29T23:26:57.843 INFO:teuthology.orchestra.run.smithi156.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-09-29T23:26:57.844 INFO:teuthology.orchestra.run.smithi156.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-09-29T23:26:57.852 INFO:teuthology.orchestra.run.smithi143.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2023-09-29T23:26:57.852 INFO:teuthology.orchestra.run.smithi143.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2023-09-29T23:26:57.852 INFO:teuthology.orchestra.run.smithi143.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2023-09-29T23:26:57.854 INFO:teuthology.orchestra.run.smithi143.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-09-29T23:26:57.854 INFO:teuthology.orchestra.run.smithi143.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2023-09-29T23:26:57.855 INFO:teuthology.orchestra.run.smithi143.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webob all 1:1.8.5-2 [84.4 kB] 2023-09-29T23:26:57.856 INFO:teuthology.orchestra.run.smithi143.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2023-09-29T23:26:57.856 INFO:teuthology.orchestra.run.smithi143.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2023-09-29T23:26:57.859 INFO:teuthology.orchestra.run.smithi143.stdout:Get:28 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2023-09-29T23:26:57.861 INFO:teuthology.orchestra.run.smithi156.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-09-29T23:26:57.863 INFO:teuthology.orchestra.run.smithi156.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-09-29T23:26:57.864 INFO:teuthology.orchestra.run.smithi156.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-09-29T23:26:57.865 INFO:teuthology.orchestra.run.smithi156.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-09-29T23:26:57.870 INFO:teuthology.orchestra.run.smithi156.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1165-g93dce5e1-1focal [3,552 kB] 2023-09-29T23:26:57.879 INFO:teuthology.orchestra.run.smithi143.stdout:Get:29 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2023-09-29T23:26:57.879 INFO:teuthology.orchestra.run.smithi143.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2023-09-29T23:26:57.885 INFO:teuthology.orchestra.run.smithi156.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-09-29T23:26:57.886 INFO:teuthology.orchestra.run.smithi156.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-09-29T23:26:57.887 INFO:teuthology.orchestra.run.smithi143.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2023-09-29T23:26:57.887 INFO:teuthology.orchestra.run.smithi143.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2023-09-29T23:26:57.888 INFO:teuthology.orchestra.run.smithi143.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2023-09-29T23:26:57.889 INFO:teuthology.orchestra.run.smithi143.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2023-09-29T23:26:57.891 INFO:teuthology.orchestra.run.smithi143.stdout:Get:35 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.1 [183 kB] 2023-09-29T23:26:57.894 INFO:teuthology.orchestra.run.smithi143.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-jinja2 all 2.10.1-2 [95.5 kB] 2023-09-29T23:26:57.895 INFO:teuthology.orchestra.run.smithi143.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-09-29T23:26:57.922 INFO:teuthology.orchestra.run.smithi143.stdout:Get:38 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2023-09-29T23:26:57.924 INFO:teuthology.orchestra.run.smithi143.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2023-09-29T23:26:57.962 INFO:teuthology.orchestra.run.smithi143.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2023-09-29T23:26:57.964 INFO:teuthology.orchestra.run.smithi143.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2023-09-29T23:26:57.965 INFO:teuthology.orchestra.run.smithi143.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2023-09-29T23:26:57.966 INFO:teuthology.orchestra.run.smithi143.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2023-09-29T23:26:58.027 INFO:teuthology.orchestra.run.smithi156.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1165-g93dce5e1-1focal [758 kB] 2023-09-29T23:26:58.054 INFO:teuthology.orchestra.run.smithi156.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1165-g93dce5e1-1focal [414 kB] 2023-09-29T23:26:58.086 INFO:teuthology.orchestra.run.smithi156.stdout:Get:47 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1165-g93dce5e1-1focal [22.5 kB] 2023-09-29T23:26:58.087 INFO:teuthology.orchestra.run.smithi156.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1165-g93dce5e1-1focal [200 kB] 2023-09-29T23:26:58.094 INFO:teuthology.orchestra.run.smithi156.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1165-g93dce5e1-1focal [47.1 kB] 2023-09-29T23:26:58.095 INFO:teuthology.orchestra.run.smithi156.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1165-g93dce5e1-1focal [385 kB] 2023-09-29T23:26:58.110 INFO:teuthology.orchestra.run.smithi156.stdout:Get:51 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1165-g93dce5e1-1focal [5,569 kB] 2023-09-29T23:26:58.110 INFO:teuthology.orchestra.run.smithi156.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-09-29T23:26:58.156 INFO:teuthology.orchestra.run.smithi156.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-09-29T23:26:58.157 INFO:teuthology.orchestra.run.smithi156.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-09-29T23:26:58.180 INFO:teuthology.orchestra.run.smithi143.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2023-09-29T23:26:58.186 INFO:teuthology.orchestra.run.smithi156.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-09-29T23:26:58.186 INFO:teuthology.orchestra.run.smithi156.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-09-29T23:26:58.186 INFO:teuthology.orchestra.run.smithi156.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-09-29T23:26:58.187 INFO:teuthology.orchestra.run.smithi156.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-09-29T23:26:58.187 INFO:teuthology.orchestra.run.smithi156.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-09-29T23:26:58.190 INFO:teuthology.orchestra.run.smithi156.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-09-29T23:26:58.190 INFO:teuthology.orchestra.run.smithi156.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-09-29T23:26:58.193 INFO:teuthology.orchestra.run.smithi143.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.6-1165-g93dce5e1-1focal [3,552 kB] 2023-09-29T23:26:58.205 INFO:teuthology.orchestra.run.smithi156.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-09-29T23:26:58.205 INFO:teuthology.orchestra.run.smithi156.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-09-29T23:26:58.206 INFO:teuthology.orchestra.run.smithi143.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2023-09-29T23:26:58.226 INFO:teuthology.orchestra.run.smithi156.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-09-29T23:26:58.231 INFO:teuthology.orchestra.run.smithi156.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-09-29T23:26:58.233 INFO:teuthology.orchestra.run.smithi156.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-09-29T23:26:58.235 INFO:teuthology.orchestra.run.smithi156.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-09-29T23:26:58.236 INFO:teuthology.orchestra.run.smithi156.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-09-29T23:26:58.239 INFO:teuthology.orchestra.run.smithi156.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-09-29T23:26:58.242 INFO:teuthology.orchestra.run.smithi156.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-09-29T23:26:58.265 INFO:teuthology.orchestra.run.smithi156.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-09-29T23:26:58.269 INFO:teuthology.orchestra.run.smithi156.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-09-29T23:26:58.274 INFO:teuthology.orchestra.run.smithi143.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2023-09-29T23:26:58.296 INFO:teuthology.orchestra.run.smithi143.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2023-09-29T23:26:58.296 INFO:teuthology.orchestra.run.smithi143.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2023-09-29T23:26:58.296 INFO:teuthology.orchestra.run.smithi143.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2023-09-29T23:26:58.297 INFO:teuthology.orchestra.run.smithi143.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2023-09-29T23:26:58.297 INFO:teuthology.orchestra.run.smithi143.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2023-09-29T23:26:58.297 INFO:teuthology.orchestra.run.smithi143.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2023-09-29T23:26:58.298 INFO:teuthology.orchestra.run.smithi143.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2023-09-29T23:26:58.299 INFO:teuthology.orchestra.run.smithi143.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2023-09-29T23:26:58.299 INFO:teuthology.orchestra.run.smithi143.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2023-09-29T23:26:58.317 INFO:teuthology.orchestra.run.smithi143.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2023-09-29T23:26:58.321 INFO:teuthology.orchestra.run.smithi143.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2023-09-29T23:26:58.330 INFO:teuthology.orchestra.run.smithi156.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-09-29T23:26:58.331 INFO:teuthology.orchestra.run.smithi156.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-09-29T23:26:58.331 INFO:teuthology.orchestra.run.smithi156.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-09-29T23:26:58.331 INFO:teuthology.orchestra.run.smithi156.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-09-29T23:26:58.332 INFO:teuthology.orchestra.run.smithi156.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-09-29T23:26:58.333 INFO:teuthology.orchestra.run.smithi156.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-09-29T23:26:58.344 INFO:teuthology.orchestra.run.smithi143.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2023-09-29T23:26:58.344 INFO:teuthology.orchestra.run.smithi156.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1165-g93dce5e1-1focal [116 kB] 2023-09-29T23:26:58.347 INFO:teuthology.orchestra.run.smithi143.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2023-09-29T23:26:58.348 INFO:teuthology.orchestra.run.smithi143.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2023-09-29T23:26:58.353 INFO:teuthology.orchestra.run.smithi143.stdout:Get:62 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2023-09-29T23:26:58.356 INFO:teuthology.orchestra.run.smithi143.stdout:Get:63 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2023-09-29T23:26:58.373 INFO:teuthology.orchestra.run.smithi143.stdout:Get:64 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.1 [283 kB] 2023-09-29T23:26:58.376 INFO:teuthology.orchestra.run.smithi143.stdout:Get:65 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2023-09-29T23:26:58.411 INFO:teuthology.orchestra.run.smithi156.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-09-29T23:26:58.412 INFO:teuthology.orchestra.run.smithi156.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1165-g93dce5e1-1focal [480 kB] 2023-09-29T23:26:58.412 INFO:teuthology.orchestra.run.smithi156.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-09-29T23:26:58.412 INFO:teuthology.orchestra.run.smithi156.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-09-29T23:26:58.413 INFO:teuthology.orchestra.run.smithi156.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-09-29T23:26:58.415 INFO:teuthology.orchestra.run.smithi156.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-09-29T23:26:58.416 INFO:teuthology.orchestra.run.smithi156.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-09-29T23:26:58.417 INFO:teuthology.orchestra.run.smithi156.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-09-29T23:26:58.417 INFO:teuthology.orchestra.run.smithi156.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-09-29T23:26:58.420 INFO:teuthology.orchestra.run.smithi156.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-09-29T23:26:58.421 INFO:teuthology.orchestra.run.smithi156.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1165-g93dce5e1-1focal [27.7 MB] 2023-09-29T23:26:58.430 INFO:teuthology.orchestra.run.smithi143.stdout:Get:66 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2023-09-29T23:26:58.431 INFO:teuthology.orchestra.run.smithi143.stdout:Get:67 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2023-09-29T23:26:58.431 INFO:teuthology.orchestra.run.smithi143.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2023-09-29T23:26:58.431 INFO:teuthology.orchestra.run.smithi143.stdout:Get:69 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2023-09-29T23:26:58.431 INFO:teuthology.orchestra.run.smithi143.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2023-09-29T23:26:58.432 INFO:teuthology.orchestra.run.smithi143.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2023-09-29T23:26:58.438 INFO:teuthology.orchestra.run.smithi156.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-09-29T23:26:58.448 INFO:teuthology.orchestra.run.smithi143.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2023-09-29T23:26:58.448 INFO:teuthology.orchestra.run.smithi143.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-09-29T23:26:58.478 INFO:teuthology.orchestra.run.smithi143.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2023-09-29T23:26:58.478 INFO:teuthology.orchestra.run.smithi143.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2023-09-29T23:26:58.480 INFO:teuthology.orchestra.run.smithi143.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2023-09-29T23:26:58.482 INFO:teuthology.orchestra.run.smithi143.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2023-09-29T23:26:58.482 INFO:teuthology.orchestra.run.smithi143.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2023-09-29T23:26:58.483 INFO:teuthology.orchestra.run.smithi143.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2023-09-29T23:26:58.486 INFO:teuthology.orchestra.run.smithi143.stdout:Get:80 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2023-09-29T23:26:58.487 INFO:teuthology.orchestra.run.smithi143.stdout:Get:81 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2023-09-29T23:26:58.577 INFO:teuthology.orchestra.run.smithi143.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.6-1165-g93dce5e1-1focal [758 kB] 2023-09-29T23:26:58.654 INFO:teuthology.orchestra.run.smithi143.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.6-1165-g93dce5e1-1focal [414 kB] 2023-09-29T23:26:58.702 INFO:teuthology.orchestra.run.smithi143.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.6-1165-g93dce5e1-1focal [22.5 kB] 2023-09-29T23:26:58.703 INFO:teuthology.orchestra.run.smithi143.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.6-1165-g93dce5e1-1focal [200 kB] 2023-09-29T23:26:58.755 INFO:teuthology.orchestra.run.smithi143.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.6-1165-g93dce5e1-1focal [47.1 kB] 2023-09-29T23:26:58.756 INFO:teuthology.orchestra.run.smithi143.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.6-1165-g93dce5e1-1focal [385 kB] 2023-09-29T23:26:58.777 INFO:teuthology.orchestra.run.smithi143.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.6-1165-g93dce5e1-1focal [5,569 kB] 2023-09-29T23:26:59.423 INFO:teuthology.orchestra.run.smithi143.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.6-1165-g93dce5e1-1focal [116 kB] 2023-09-29T23:26:59.464 INFO:teuthology.orchestra.run.smithi143.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.6-1165-g93dce5e1-1focal [480 kB] 2023-09-29T23:26:59.490 INFO:teuthology.orchestra.run.smithi143.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.6-1165-g93dce5e1-1focal [27.7 MB] 2023-09-29T23:26:59.797 INFO:teuthology.orchestra.run.smithi156.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1165-g93dce5e1-1focal [6,453 kB] 2023-09-29T23:26:59.961 INFO:teuthology.orchestra.run.smithi156.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1165-g93dce5e1-1focal [221 kB] 2023-09-29T23:26:59.967 INFO:teuthology.orchestra.run.smithi156.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1165-g93dce5e1-1focal [153 kB] 2023-09-29T23:26:59.982 INFO:teuthology.orchestra.run.smithi156.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1165-g93dce5e1-1focal [1,589 kB] 2023-09-29T23:27:00.054 INFO:teuthology.orchestra.run.smithi156.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1165-g93dce5e1-1focal [7,845 kB] 2023-09-29T23:27:00.417 INFO:teuthology.orchestra.run.smithi156.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1165-g93dce5e1-1focal [28.7 MB] 2023-09-29T23:27:01.846 INFO:teuthology.orchestra.run.smithi156.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1165-g93dce5e1-1focal [3,928 B] 2023-09-29T23:27:01.847 INFO:teuthology.orchestra.run.smithi156.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1165-g93dce5e1-1focal [868 kB] 2023-09-29T23:27:01.859 INFO:teuthology.orchestra.run.smithi156.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1165-g93dce5e1-1focal [2,404 kB] 2023-09-29T23:27:01.893 INFO:teuthology.orchestra.run.smithi156.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1165-g93dce5e1-1focal [78.9 kB] 2023-09-29T23:27:01.896 INFO:teuthology.orchestra.run.smithi156.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1165-g93dce5e1-1focal [117 kB] 2023-09-29T23:27:01.900 INFO:teuthology.orchestra.run.smithi156.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1165-g93dce5e1-1focal [1,760 kB] 2023-09-29T23:27:01.980 INFO:teuthology.orchestra.run.smithi156.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1165-g93dce5e1-1focal [7,861 kB] 2023-09-29T23:27:02.345 INFO:teuthology.orchestra.run.smithi156.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1165-g93dce5e1-1focal [4,048 B] 2023-09-29T23:27:02.345 INFO:teuthology.orchestra.run.smithi156.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1165-g93dce5e1-1focal [56.2 MB] 2023-09-29T23:27:02.703 INFO:teuthology.orchestra.run.smithi143.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.6-1165-g93dce5e1-1focal [6,453 kB] 2023-09-29T23:27:03.585 INFO:teuthology.orchestra.run.smithi143.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.6-1165-g93dce5e1-1focal [221 kB] 2023-09-29T23:27:03.624 INFO:teuthology.orchestra.run.smithi143.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.6-1165-g93dce5e1-1focal [153 kB] 2023-09-29T23:27:03.655 INFO:teuthology.orchestra.run.smithi143.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.6-1165-g93dce5e1-1focal [1,589 kB] 2023-09-29T23:27:03.861 INFO:teuthology.orchestra.run.smithi143.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.6-1165-g93dce5e1-1focal [7,845 kB] 2023-09-29T23:27:04.917 INFO:teuthology.orchestra.run.smithi143.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.6-1165-g93dce5e1-1focal [28.7 MB] 2023-09-29T23:27:04.967 INFO:teuthology.orchestra.run.smithi156.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1165-g93dce5e1-1focal [229 kB] 2023-09-29T23:27:04.972 INFO:teuthology.orchestra.run.smithi156.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1165-g93dce5e1-1focal [21.9 kB] 2023-09-29T23:27:05.038 INFO:teuthology.orchestra.run.smithi156.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1165-g93dce5e1-1focal [17.0 MB] 2023-09-29T23:27:05.730 INFO:teuthology.orchestra.run.smithi156.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1165-g93dce5e1-1focal [79.5 kB] 2023-09-29T23:27:06.354 INFO:teuthology.orchestra.run.smithi156.stdout:Fetched 209 MB in 8s (25.4 MB/s) 2023-09-29T23:27:06.452 INFO:teuthology.orchestra.run.smithi143.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.6-1165-g93dce5e1-1focal [3,928 B] 2023-09-29T23:27:06.517 INFO:teuthology.orchestra.run.smithi143.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.6-1165-g93dce5e1-1focal [868 kB] 2023-09-29T23:27:06.531 INFO:teuthology.orchestra.run.smithi143.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.6-1165-g93dce5e1-1focal [2,404 kB] 2023-09-29T23:27:06.532 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-09-29T23:27:06.579 INFO:teuthology.orchestra.run.smithi143.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.6-1165-g93dce5e1-1focal [78.9 kB] 2023-09-29T23:27:06.580 INFO:teuthology.orchestra.run.smithi143.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.6-1165-g93dce5e1-1focal [117 kB] 2023-09-29T23:27:06.582 INFO:teuthology.orchestra.run.smithi143.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.6-1165-g93dce5e1-1focal [1,760 kB] 2023-09-29T23:27:06.591 INFO:teuthology.orchestra.run.smithi156.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141758 files and directories currently installed.) 2023-09-29T23:27:06.595 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-09-29T23:27:06.646 INFO:teuthology.orchestra.run.smithi143.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.6-1165-g93dce5e1-1focal [7,861 kB] 2023-09-29T23:27:06.675 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-29T23:27:06.933 INFO:teuthology.orchestra.run.smithi143.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.6-1165-g93dce5e1-1focal [4,048 B] 2023-09-29T23:27:06.933 INFO:teuthology.orchestra.run.smithi143.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.6-1165-g93dce5e1-1focal [56.2 MB] 2023-09-29T23:27:07.049 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../001-librbd1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:07.171 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking librbd1 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-29T23:27:07.884 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../002-librados2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:08.034 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking librados2 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-29T23:27:08.680 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libcephfs2. 2023-09-29T23:27:08.696 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../003-libcephfs2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:08.738 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:09.107 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-rados. 2023-09-29T23:27:09.123 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../004-python3-rados_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:09.149 INFO:teuthology.orchestra.run.smithi143.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.6-1165-g93dce5e1-1focal [229 kB] 2023-09-29T23:27:09.153 INFO:teuthology.orchestra.run.smithi143.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.6-1165-g93dce5e1-1focal [21.9 kB] 2023-09-29T23:27:09.174 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:09.222 INFO:teuthology.orchestra.run.smithi143.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.6-1165-g93dce5e1-1focal [17.0 MB] 2023-09-29T23:27:09.535 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-09-29T23:27:09.554 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:09.593 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:09.761 INFO:teuthology.orchestra.run.smithi143.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri4-testing-2023-09-29-0928-quincy/93dce5e17918f8aca65993a7a2dc40233a47b481/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.6-1165-g93dce5e1-1focal [79.5 kB] 2023-09-29T23:27:09.795 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-cephfs. 2023-09-29T23:27:09.812 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../006-python3-cephfs_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:09.845 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:10.173 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-ceph-common. 2023-09-29T23:27:10.189 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:10.231 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:10.384 INFO:teuthology.orchestra.run.smithi143.stdout:Fetched 209 MB in 12s (17.1 MB/s) 2023-09-29T23:27:10.554 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2023-09-29T23:27:10.576 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-prettytable. 2023-09-29T23:27:10.592 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2023-09-29T23:27:10.609 INFO:teuthology.orchestra.run.smithi143.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141758 files and directories currently installed.) 2023-09-29T23:27:10.614 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2023-09-29T23:27:10.634 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-09-29T23:27:10.711 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-29T23:27:10.912 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-rbd. 2023-09-29T23:27:10.928 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../009-python3-rbd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:10.970 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:11.006 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../001-librbd1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:11.132 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librbd1 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-29T23:27:11.273 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-09-29T23:27:11.289 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-09-29T23:27:11.323 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-29T23:27:11.693 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-09-29T23:27:11.708 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2023-09-29T23:27:11.751 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-09-29T23:27:11.837 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../002-librados2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:11.962 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librados2 (17.2.6-1165-g93dce5e1-1focal) over (15.2.17-0ubuntu0.20.04.4) ... 2023-09-29T23:27:12.104 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-09-29T23:27:12.122 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-09-29T23:27:12.162 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-29T23:27:12.531 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-09-29T23:27:12.547 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-09-29T23:27:12.590 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-29T23:27:12.616 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libcephfs2. 2023-09-29T23:27:12.632 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../003-libcephfs2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:12.691 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:12.952 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-09-29T23:27:12.968 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-09-29T23:27:12.993 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rados. 2023-09-29T23:27:13.011 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-29T23:27:13.011 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../004-python3-rados_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:13.051 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:13.437 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-09-29T23:27:13.453 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:13.495 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:13.507 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-09-29T23:27:13.524 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-29T23:27:13.575 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:27:13.773 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cephfs. 2023-09-29T23:27:13.789 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../006-python3-cephfs_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:13.831 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:14.106 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-09-29T23:27:14.123 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-29T23:27:14.164 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:27:14.175 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-ceph-common. 2023-09-29T23:27:14.194 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:14.225 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:14.543 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-09-29T23:27:14.560 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-29T23:27:14.601 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:27:14.778 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-prettytable. 2023-09-29T23:27:14.794 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2023-09-29T23:27:14.828 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2023-09-29T23:27:15.065 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rbd. 2023-09-29T23:27:15.084 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../009-python3-rbd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:15.122 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:15.305 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-09-29T23:27:15.320 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-09-29T23:27:15.363 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-29T23:27:15.483 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2023-09-29T23:27:15.503 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2023-09-29T23:27:15.566 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-29T23:27:15.757 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package librgw2. 2023-09-29T23:27:15.774 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../019-librgw2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:15.807 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:15.927 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librabbitmq4:amd64. 2023-09-29T23:27:15.944 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2023-09-29T23:27:15.985 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2023-09-29T23:27:16.313 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-09-29T23:27:16.332 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2023-09-29T23:27:16.363 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-29T23:27:16.527 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-rgw. 2023-09-29T23:27:16.544 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../020-python3-rgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:16.585 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:16.716 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-09-29T23:27:16.735 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2023-09-29T23:27:16.774 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-29T23:27:16.963 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package liboath0. 2023-09-29T23:27:16.980 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2023-09-29T23:27:17.021 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-09-29T23:27:17.099 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-09-29T23:27:17.115 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2023-09-29T23:27:17.157 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-29T23:27:17.357 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libradosstriper1. 2023-09-29T23:27:17.373 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../022-libradosstriper1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:17.415 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:17.598 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-09-29T23:27:17.616 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-29T23:27:17.658 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:27:17.827 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-common. 2023-09-29T23:27:17.846 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../023-ceph-common_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:17.885 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:18.123 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-09-29T23:27:18.140 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-29T23:27:18.181 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:27:18.525 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-09-29T23:27:18.544 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2023-09-29T23:27:18.584 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:27:19.021 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libthrift-0.13.0. 2023-09-29T23:27:19.037 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2023-09-29T23:27:19.078 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-29T23:27:19.440 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package librgw2. 2023-09-29T23:27:19.459 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../019-librgw2_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:19.498 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:20.200 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rgw. 2023-09-29T23:27:20.217 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../020-python3-rgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:20.250 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:20.553 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liboath0. 2023-09-29T23:27:20.557 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-base. 2023-09-29T23:27:20.569 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2023-09-29T23:27:20.575 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../024-ceph-base_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:20.611 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2023-09-29T23:27:20.651 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:20.947 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libradosstriper1. 2023-09-29T23:27:20.964 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../022-libradosstriper1_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:21.006 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:21.492 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-common. 2023-09-29T23:27:21.509 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../023-ceph-common_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:21.550 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:21.620 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-cryptography. 2023-09-29T23:27:21.636 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-09-29T23:27:21.703 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-29T23:27:22.043 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-openssl. 2023-09-29T23:27:22.062 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2023-09-29T23:27:22.101 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-09-29T23:27:22.396 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-09-29T23:27:22.412 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:22.454 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:22.834 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-bcrypt. 2023-09-29T23:27:22.854 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-09-29T23:27:22.892 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-29T23:27:23.212 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-cherrypy3. 2023-09-29T23:27:23.228 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-6_all.deb ... 2023-09-29T23:27:23.270 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-09-29T23:27:23.615 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-natsort. 2023-09-29T23:27:23.634 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../030-python3-natsort_7.0.1-1_all.deb ... 2023-09-29T23:27:23.673 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-09-29T23:27:24.002 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-jwt. 2023-09-29T23:27:24.020 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../031-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-09-29T23:27:24.085 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-29T23:27:24.319 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-base. 2023-09-29T23:27:24.338 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../024-ceph-base_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:24.380 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-logutils. 2023-09-29T23:27:24.398 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-7_all.deb ... 2023-09-29T23:27:24.399 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:24.438 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-09-29T23:27:24.767 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-markupsafe. 2023-09-29T23:27:24.783 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../033-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-09-29T23:27:24.825 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-09-29T23:27:25.161 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-mako. 2023-09-29T23:27:25.177 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../034-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-09-29T23:27:25.220 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-29T23:27:25.384 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cryptography. 2023-09-29T23:27:25.403 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.1_amd64.deb ... 2023-09-29T23:27:25.442 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-29T23:27:25.566 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-simplegeneric. 2023-09-29T23:27:25.582 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-3_all.deb ... 2023-09-29T23:27:25.623 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-09-29T23:27:25.821 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-openssl. 2023-09-29T23:27:25.839 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2023-09-29T23:27:25.859 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-singledispatch. 2023-09-29T23:27:25.871 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2023-09-29T23:27:25.878 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-09-29T23:27:25.917 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-09-29T23:27:26.141 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-09-29T23:27:26.157 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../027-ceph-mgr-modules-core_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:26.199 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:26.262 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-webob. 2023-09-29T23:27:26.280 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../037-python3-webob_1%3a1.8.5-2_all.deb ... 2023-09-29T23:27:26.320 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-09-29T23:27:26.587 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-bcrypt. 2023-09-29T23:27:26.606 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../028-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2023-09-29T23:27:26.645 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-29T23:27:26.657 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-soupsieve. 2023-09-29T23:27:26.676 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../038-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-09-29T23:27:26.715 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-29T23:27:26.981 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cherrypy3. 2023-09-29T23:27:26.997 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../029-python3-cherrypy3_8.9.1-6_all.deb ... 2023-09-29T23:27:27.039 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2023-09-29T23:27:27.043 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-bs4. 2023-09-29T23:27:27.060 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../039-python3-bs4_4.8.2-1_all.deb ... 2023-09-29T23:27:27.101 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-09-29T23:27:27.409 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-natsort. 2023-09-29T23:27:27.425 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../030-python3-natsort_7.0.1-1_all.deb ... 2023-09-29T23:27:27.467 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-natsort (7.0.1-1) ... 2023-09-29T23:27:27.488 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-waitress. 2023-09-29T23:27:27.504 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../040-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-09-29T23:27:27.549 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-29T23:27:27.804 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jwt. 2023-09-29T23:27:27.821 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../031-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2023-09-29T23:27:27.862 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-29T23:27:27.874 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-tempita. 2023-09-29T23:27:27.892 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-6_all.deb ... 2023-09-29T23:27:27.941 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-09-29T23:27:28.206 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-logutils. 2023-09-29T23:27:28.222 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-7_all.deb ... 2023-09-29T23:27:28.252 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-paste. 2023-09-29T23:27:28.265 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-logutils (0.3.3-7) ... 2023-09-29T23:27:28.269 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../042-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-09-29T23:27:28.311 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-09-29T23:27:28.567 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-markupsafe. 2023-09-29T23:27:28.584 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../033-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2023-09-29T23:27:28.650 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2023-09-29T23:27:28.656 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-09-29T23:27:28.672 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-09-29T23:27:28.714 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-09-29T23:27:28.945 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-mako. 2023-09-29T23:27:28.961 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../034-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2023-09-29T23:27:29.003 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-29T23:27:29.042 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pastedeploy. 2023-09-29T23:27:29.058 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../044-python3-pastedeploy_2.1.0-1_all.deb ... 2023-09-29T23:27:29.100 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-09-29T23:27:29.348 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-simplegeneric. 2023-09-29T23:27:29.367 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-3_all.deb ... 2023-09-29T23:27:29.395 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-webtest. 2023-09-29T23:27:29.406 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-09-29T23:27:29.414 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../045-python3-webtest_2.0.32-2_all.deb ... 2023-09-29T23:27:29.453 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-09-29T23:27:29.742 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-singledispatch. 2023-09-29T23:27:29.760 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2023-09-29T23:27:29.764 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pecan. 2023-09-29T23:27:29.780 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../046-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-09-29T23:27:29.801 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2023-09-29T23:27:29.847 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-09-29T23:27:30.211 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-werkzeug. 2023-09-29T23:27:30.228 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../047-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-09-29T23:27:30.269 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-29T23:27:30.845 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-webob. 2023-09-29T23:27:30.862 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../037-python3-webob_1%3a1.8.5-2_all.deb ... 2023-09-29T23:27:30.895 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-webob (1:1.8.5-2) ... 2023-09-29T23:27:31.190 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-soupsieve. 2023-09-29T23:27:31.208 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../038-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2023-09-29T23:27:31.239 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-29T23:27:31.473 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-09-29T23:27:31.483 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-bs4. 2023-09-29T23:27:31.491 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:31.501 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../039-python3-bs4_4.8.2-1_all.deb ... 2023-09-29T23:27:31.531 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:31.542 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2023-09-29T23:27:31.893 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr. 2023-09-29T23:27:31.911 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../049-ceph-mgr_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:31.911 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-waitress. 2023-09-29T23:27:31.929 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../040-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2023-09-29T23:27:31.950 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:31.973 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-29T23:27:32.323 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-tempita. 2023-09-29T23:27:32.342 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-6_all.deb ... 2023-09-29T23:27:32.381 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-tempita (0.5.2-6) ... 2023-09-29T23:27:32.487 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mon. 2023-09-29T23:27:32.504 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../050-ceph-mon_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:32.545 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:32.683 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-paste. 2023-09-29T23:27:32.699 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../042-python3-paste_3.4.0+dfsg1-1_all.deb ... 2023-09-29T23:27:32.741 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2023-09-29T23:27:33.078 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-09-29T23:27:33.096 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2023-09-29T23:27:33.136 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2023-09-29T23:27:33.424 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pastedeploy. 2023-09-29T23:27:33.443 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../044-python3-pastedeploy_2.1.0-1_all.deb ... 2023-09-29T23:27:33.474 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2023-09-29T23:27:33.724 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-osd. 2023-09-29T23:27:33.740 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../051-ceph-osd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:33.782 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:33.786 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-webtest. 2023-09-29T23:27:33.801 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../045-python3-webtest_2.0.32-2_all.deb ... 2023-09-29T23:27:33.844 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-webtest (2.0.32-2) ... 2023-09-29T23:27:34.189 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pecan. 2023-09-29T23:27:34.205 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../046-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2023-09-29T23:27:34.247 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2023-09-29T23:27:34.627 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-werkzeug. 2023-09-29T23:27:34.644 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../047-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.1_all.deb ... 2023-09-29T23:27:34.685 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-29T23:27:35.064 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-09-29T23:27:35.081 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:35.122 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:35.517 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr. 2023-09-29T23:27:35.533 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../049-ceph-mgr_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:35.575 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:36.095 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mon. 2023-09-29T23:27:36.115 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../050-ceph-mon_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:36.145 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:36.644 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph. 2023-09-29T23:27:36.661 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../052-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:36.702 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:37.038 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-fuse. 2023-09-29T23:27:37.058 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../053-ceph-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:37.096 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:37.299 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-osd. 2023-09-29T23:27:37.316 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../051-ceph-osd_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:37.377 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mds. 2023-09-29T23:27:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../054-ceph-mds_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:38.203 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package cephadm. 2023-09-29T23:27:38.218 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../055-cephadm_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:38.261 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:38.597 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-jinja2. 2023-09-29T23:27:38.613 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2_all.deb ... 2023-09-29T23:27:38.655 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-09-29T23:27:38.966 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-09-29T23:27:38.982 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:39.024 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:39.395 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-repoze.lru. 2023-09-29T23:27:39.410 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-2_all.deb ... 2023-09-29T23:27:39.453 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-09-29T23:27:39.689 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libjs-underscore. 2023-09-29T23:27:39.705 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../059-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-09-29T23:27:39.747 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-29T23:27:40.033 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-09-29T23:27:40.050 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../060-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-09-29T23:27:40.091 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-29T23:27:40.169 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph. 2023-09-29T23:27:40.185 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../052-ceph_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:40.227 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:40.435 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-routes. 2023-09-29T23:27:40.455 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../061-python3-routes_2.4.1-2_all.deb ... 2023-09-29T23:27:40.493 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-09-29T23:27:40.555 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-fuse. 2023-09-29T23:27:40.570 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../053-ceph-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:40.612 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:40.847 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-09-29T23:27:40.864 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../062-ceph-mgr-dashboard_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:40.905 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:41.166 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mds. 2023-09-29T23:27:41.185 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../054-ceph-mds_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:41.515 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:41.515 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-decorator. 2023-09-29T23:27:41.515 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../063-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-09-29T23:27:41.517 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-09-29T23:27:41.754 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package cephadm. 2023-09-29T23:27:41.771 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../055-cephadm_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:41.828 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-09-29T23:27:41.835 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:41.845 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../064-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-09-29T23:27:41.878 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-29T23:27:42.139 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-jinja2. 2023-09-29T23:27:42.154 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../056-python3-jinja2_2.10.1-2_all.deb ... 2023-09-29T23:27:42.197 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-jinja2 (2.10.1-2) ... 2023-09-29T23:27:42.231 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-scipy. 2023-09-29T23:27:42.247 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../065-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-09-29T23:27:42.289 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-09-29T23:27:42.525 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-09-29T23:27:42.542 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:42.583 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:42.945 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-repoze.lru. 2023-09-29T23:27:42.961 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-2_all.deb ... 2023-09-29T23:27:43.003 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-09-29T23:27:43.289 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libjs-underscore. 2023-09-29T23:27:43.306 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../059-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2023-09-29T23:27:43.347 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-29T23:27:43.549 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-09-29T23:27:43.565 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../066-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-09-29T23:27:43.606 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-29T23:27:43.633 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libjs-sphinxdoc. 2023-09-29T23:27:43.650 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../060-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2023-09-29T23:27:43.691 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-29T23:27:44.019 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-routes. 2023-09-29T23:27:44.039 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../061-python3-routes_2.4.1-2_all.deb ... 2023-09-29T23:27:44.077 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-routes (2.4.1-2) ... 2023-09-29T23:27:44.118 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-joblib. 2023-09-29T23:27:44.138 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../067-python3-joblib_0.14.0-3_all.deb ... 2023-09-29T23:27:44.202 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-09-29T23:27:44.430 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-09-29T23:27:44.448 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../062-ceph-mgr-dashboard_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:44.489 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:44.564 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-sklearn. 2023-09-29T23:27:44.580 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../068-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-09-29T23:27:44.647 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-29T23:27:45.009 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-decorator. 2023-09-29T23:27:45.025 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../063-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2023-09-29T23:27:45.067 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2023-09-29T23:27:45.389 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-09-29T23:27:45.395 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2023-09-29T23:27:45.405 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../069-ceph-mgr-diskprediction-local_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:45.412 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../064-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2023-09-29T23:27:45.449 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:45.453 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-29T23:27:45.756 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-scipy. 2023-09-29T23:27:45.773 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../065-python3-scipy_1.3.3-3build1_amd64.deb ... 2023-09-29T23:27:45.832 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2023-09-29T23:27:46.942 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-cachetools. 2023-09-29T23:27:46.959 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../070-python3-cachetools_4.0.0-1_all.deb ... 2023-09-29T23:27:47.000 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-09-29T23:27:47.345 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pyasn1. 2023-09-29T23:27:47.361 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-09-29T23:27:47.403 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-09-29T23:27:47.433 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-sklearn-lib. 2023-09-29T23:27:47.450 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../066-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2023-09-29T23:27:47.491 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-29T23:27:47.732 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-09-29T23:27:47.843 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-09-29T23:27:47.843 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-29T23:27:48.094 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-rsa. 2023-09-29T23:27:48.111 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-joblib. 2023-09-29T23:27:48.114 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../073-python3-rsa_4.0-3_all.deb ... 2023-09-29T23:27:48.127 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../067-python3-joblib_0.14.0-3_all.deb ... 2023-09-29T23:27:48.151 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-rsa (4.0-3) ... 2023-09-29T23:27:48.169 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-joblib (0.14.0-3) ... 2023-09-29T23:27:48.430 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-google-auth. 2023-09-29T23:27:48.449 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../074-python3-google-auth_1.5.1-2_all.deb ... 2023-09-29T23:27:48.488 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-09-29T23:27:48.542 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-sklearn. 2023-09-29T23:27:48.560 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../068-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2023-09-29T23:27:48.606 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-29T23:27:48.849 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-blinker. 2023-09-29T23:27:48.870 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../075-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-09-29T23:27:48.907 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-29T23:27:49.227 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-oauthlib. 2023-09-29T23:27:49.246 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../076-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-09-29T23:27:49.285 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-29T23:27:49.374 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-09-29T23:27:49.390 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../069-ceph-mgr-diskprediction-local_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:49.432 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:49.606 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-09-29T23:27:49.622 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../077-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-09-29T23:27:49.655 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-29T23:27:49.967 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-websocket. 2023-09-29T23:27:49.984 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../078-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-09-29T23:27:50.050 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-09-29T23:27:50.378 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-kubernetes. 2023-09-29T23:27:50.394 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../079-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-09-29T23:27:50.436 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-09-29T23:27:50.568 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-cachetools. 2023-09-29T23:27:50.587 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../070-python3-cachetools_4.0.0-1_all.deb ... 2023-09-29T23:27:50.626 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2023-09-29T23:27:50.904 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyasn1. 2023-09-29T23:27:50.921 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3build1_all.deb ... 2023-09-29T23:27:50.979 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2023-09-29T23:27:51.066 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-09-29T23:27:51.084 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../080-ceph-mgr-k8sevents_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:51.116 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:51.258 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-09-29T23:27:51.276 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2023-09-29T23:27:51.307 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-29T23:27:51.555 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libonig5:amd64. 2023-09-29T23:27:51.572 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../081-libonig5_6.9.4-1_amd64.deb ... 2023-09-29T23:27:51.602 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-rsa. 2023-09-29T23:27:51.613 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-09-29T23:27:51.620 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../073-python3-rsa_4.0-3_all.deb ... 2023-09-29T23:27:51.660 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-rsa (4.0-3) ... 2023-09-29T23:27:51.989 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-google-auth. 2023-09-29T23:27:51.999 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libjq1:amd64. 2023-09-29T23:27:52.007 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../074-python3-google-auth_1.5.1-2_all.deb ... 2023-09-29T23:27:52.017 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../082-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-29T23:27:52.047 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2023-09-29T23:27:52.057 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-29T23:27:52.335 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package jq. 2023-09-29T23:27:52.353 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../083-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-29T23:27:52.401 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-blinker. 2023-09-29T23:27:52.413 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-09-29T23:27:52.420 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../075-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2023-09-29T23:27:52.458 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-29T23:27:52.680 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package socat. 2023-09-29T23:27:52.698 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../084-socat_1.7.3.3-2_amd64.deb ... 2023-09-29T23:27:52.738 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking socat (1.7.3.3-2) ... 2023-09-29T23:27:52.761 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-oauthlib. 2023-09-29T23:27:52.780 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../076-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2023-09-29T23:27:52.811 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-29T23:27:53.075 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package xmlstarlet. 2023-09-29T23:27:53.093 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-09-29T23:27:53.133 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-09-29T23:27:53.165 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-09-29T23:27:53.181 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../077-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2023-09-29T23:27:53.223 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-29T23:27:53.481 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-test. 2023-09-29T23:27:53.497 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../086-ceph-test_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:53.538 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:53.559 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-websocket. 2023-09-29T23:27:53.575 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../078-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2023-09-29T23:27:53.617 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2023-09-29T23:27:53.954 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-kubernetes. 2023-09-29T23:27:53.972 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../079-python3-kubernetes_7.0.0~a1-2_all.deb ... 2023-09-29T23:27:54.012 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2023-09-29T23:27:54.567 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-09-29T23:27:54.583 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../080-ceph-mgr-k8sevents_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:54.625 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:55.056 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libonig5:amd64. 2023-09-29T23:27:55.073 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../081-libonig5_6.9.4-1_amd64.deb ... 2023-09-29T23:27:55.114 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2023-09-29T23:27:55.484 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libjq1:amd64. 2023-09-29T23:27:55.503 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../082-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-29T23:27:55.542 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-29T23:27:55.845 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package jq. 2023-09-29T23:27:55.863 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../083-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2023-09-29T23:27:55.903 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2023-09-29T23:27:56.156 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package socat. 2023-09-29T23:27:56.173 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../084-socat_1.7.3.3-2_amd64.deb ... 2023-09-29T23:27:56.239 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking socat (1.7.3.3-2) ... 2023-09-29T23:27:56.543 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package xmlstarlet. 2023-09-29T23:27:56.559 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2build1_amd64.deb ... 2023-09-29T23:27:56.601 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2023-09-29T23:27:56.965 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-test. 2023-09-29T23:27:56.984 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../086-ceph-test_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:57.023 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:58.551 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package ceph-volume. 2023-09-29T23:27:58.570 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../087-ceph-volume_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:27:58.609 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:27:58.972 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package fonts-mathjax. 2023-09-29T23:27:58.990 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../088-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-29T23:27:59.030 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-29T23:27:59.551 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libcephfs-dev. 2023-09-29T23:27:59.568 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../089-libcephfs-dev_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:27:59.609 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:00.012 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package nvme-cli. 2023-09-29T23:28:00.029 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-09-29T23:28:00.086 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-09-29T23:28:00.450 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package libjs-mathjax. 2023-09-29T23:28:00.468 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../091-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-29T23:28:00.508 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-29T23:28:02.094 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package ceph-volume. 2023-09-29T23:28:02.110 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../087-ceph-volume_17.2.6-1165-g93dce5e1-1focal_all.deb ... 2023-09-29T23:28:02.143 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:02.306 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python-natsort-doc. 2023-09-29T23:28:02.325 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../092-python-natsort-doc_7.0.1-1_all.deb ... 2023-09-29T23:28:02.364 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-09-29T23:28:02.457 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package fonts-mathjax. 2023-09-29T23:28:02.475 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../088-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-29T23:28:02.566 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-29T23:28:02.726 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-atomicwrites. 2023-09-29T23:28:02.744 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../093-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-09-29T23:28:02.783 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-09-29T23:28:03.069 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libcephfs-dev. 2023-09-29T23:28:03.088 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../089-libcephfs-dev_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:28:03.120 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-attr. 2023-09-29T23:28:03.127 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:03.140 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../094-python3-attr_19.3.0-2_all.deb ... 2023-09-29T23:28:03.178 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-09-29T23:28:03.521 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package nvme-cli. 2023-09-29T23:28:03.531 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-webencodings. 2023-09-29T23:28:03.539 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../090-nvme-cli_1.9-1ubuntu0.1_amd64.deb ... 2023-09-29T23:28:03.549 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../095-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-09-29T23:28:03.599 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking nvme-cli (1.9-1ubuntu0.1) ... 2023-09-29T23:28:03.614 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-29T23:28:03.919 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package libjs-mathjax. 2023-09-29T23:28:03.925 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-html5lib. 2023-09-29T23:28:03.937 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../091-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2023-09-29T23:28:03.947 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../096-python3-html5lib_1.0.1-2_all.deb ... 2023-09-29T23:28:03.983 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-09-29T23:28:03.998 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-29T23:28:04.345 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-09-29T23:28:04.364 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../097-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-09-29T23:28:04.403 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-29T23:28:04.907 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-packaging. 2023-09-29T23:28:04.925 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../098-python3-packaging_20.3-1_all.deb ... 2023-09-29T23:28:04.965 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-packaging (20.3-1) ... 2023-09-29T23:28:05.310 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pastescript. 2023-09-29T23:28:05.331 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2023-09-29T23:28:05.368 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-09-29T23:28:05.710 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python-natsort-doc. 2023-09-29T23:28:05.721 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pluggy. 2023-09-29T23:28:05.730 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../092-python-natsort-doc_7.0.1-1_all.deb ... 2023-09-29T23:28:05.741 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2023-09-29T23:28:05.768 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2023-09-29T23:28:05.779 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-09-29T23:28:06.116 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-psutil. 2023-09-29T23:28:06.130 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-atomicwrites. 2023-09-29T23:28:06.133 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-09-29T23:28:06.146 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../093-python3-atomicwrites_1.1.5-2build1_all.deb ... 2023-09-29T23:28:06.182 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-09-29T23:28:06.187 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2023-09-29T23:28:06.499 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-attr. 2023-09-29T23:28:06.517 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../094-python3-attr_19.3.0-2_all.deb ... 2023-09-29T23:28:06.544 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-py. 2023-09-29T23:28:06.557 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-attr (19.3.0-2) ... 2023-09-29T23:28:06.561 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-09-29T23:28:06.602 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-09-29T23:28:06.885 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-webencodings. 2023-09-29T23:28:06.902 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../095-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2023-09-29T23:28:06.914 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pyinotify. 2023-09-29T23:28:06.932 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-09-29T23:28:06.976 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-29T23:28:07.005 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-29T23:28:07.246 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-html5lib. 2023-09-29T23:28:07.263 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../096-python3-html5lib_1.0.1-2_all.deb ... 2023-09-29T23:28:07.296 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2023-09-29T23:28:07.325 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-wcwidth. 2023-09-29T23:28:07.342 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-09-29T23:28:07.383 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-29T23:28:07.675 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-lxml:amd64. 2023-09-29T23:28:07.695 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../097-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2023-09-29T23:28:07.702 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-pytest. 2023-09-29T23:28:07.723 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2023-09-29T23:28:07.732 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-29T23:28:07.760 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-09-29T23:28:08.115 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package python3-simplejson. 2023-09-29T23:28:08.135 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-09-29T23:28:08.173 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-29T23:28:08.211 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-packaging. 2023-09-29T23:28:08.231 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../098-python3-packaging_20.3-1_all.deb ... 2023-09-29T23:28:08.269 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-packaging (20.3-1) ... 2023-09-29T23:28:08.484 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-09-29T23:28:08.501 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-09-29T23:28:08.542 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-29T23:28:08.597 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pastescript. 2023-09-29T23:28:08.616 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2023-09-29T23:28:08.655 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-09-29T23:28:09.001 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pluggy. 2023-09-29T23:28:09.020 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2023-09-29T23:28:09.059 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2023-09-29T23:28:09.288 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package radosgw. 2023-09-29T23:28:09.305 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../108-radosgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:28:09.346 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:09.379 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-psutil. 2023-09-29T23:28:09.400 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2023-09-29T23:28:09.445 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2023-09-29T23:28:09.748 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-py. 2023-09-29T23:28:09.766 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2023-09-29T23:28:09.798 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2023-09-29T23:28:10.126 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pyinotify. 2023-09-29T23:28:10.143 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2023-09-29T23:28:10.184 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-29T23:28:10.538 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-wcwidth. 2023-09-29T23:28:10.554 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2023-09-29T23:28:10.612 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-29T23:28:10.924 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-pytest. 2023-09-29T23:28:10.944 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2023-09-29T23:28:10.981 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-pytest (4.6.9-1) ... 2023-09-29T23:28:11.076 INFO:teuthology.orchestra.run.smithi156.stdout:Selecting previously unselected package rbd-fuse. 2023-09-29T23:28:11.093 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing to unpack .../109-rbd-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:28:11.134 INFO:teuthology.orchestra.run.smithi156.stdout:Unpacking rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:11.344 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package python3-simplejson. 2023-09-29T23:28:11.361 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2023-09-29T23:28:11.402 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-29T23:28:11.524 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-attr (19.3.0-2) ... 2023-09-29T23:28:11.656 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-09-29T23:28:11.676 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2023-09-29T23:28:11.706 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-29T23:28:11.864 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-09-29T23:28:11.990 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-29T23:28:12.108 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-09-29T23:28:12.338 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:12.527 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package radosgw. 2023-09-29T23:28:12.543 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../108-radosgw_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:28:12.585 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:12.984 INFO:teuthology.orchestra.run.smithi156.stdout:Adding system user cephadm....done 2023-09-29T23:28:13.145 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-29T23:28:13.304 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-29T23:28:13.648 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-09-29T23:28:13.924 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-29T23:28:14.042 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-09-29T23:28:14.168 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-09-29T23:28:14.388 INFO:teuthology.orchestra.run.smithi143.stdout:Selecting previously unselected package rbd-fuse. 2023-09-29T23:28:14.406 INFO:teuthology.orchestra.run.smithi143.stdout:Preparing to unpack .../109-rbd-fuse_17.2.6-1165-g93dce5e1-1focal_amd64.deb ... 2023-09-29T23:28:14.446 INFO:teuthology.orchestra.run.smithi143.stdout:Unpacking rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:14.528 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-09-29T23:28:14.768 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-attr (19.3.0-2) ... 2023-09-29T23:28:14.939 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-29T23:28:15.061 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2023-09-29T23:28:15.204 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2023-09-29T23:28:15.240 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-09-29T23:28:15.330 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up nvme-cli (1.9-1ubuntu0.1) ... 2023-09-29T23:28:15.517 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-29T23:28:15.531 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:15.797 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-29T23:28:16.090 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:16.172 INFO:teuthology.orchestra.run.smithi143.stdout:Adding system user cephadm....done 2023-09-29T23:28:16.334 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2023-09-29T23:28:16.407 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-29T23:28:16.460 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2023-09-29T23:28:16.525 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-09-29T23:28:16.819 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-29T23:28:16.820 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-09-29T23:28:17.136 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-29T23:28:17.263 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-09-29T23:28:17.496 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2023-09-29T23:28:17.647 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liboath0 (2.6.1-1.3) ... 2023-09-29T23:28:17.773 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2023-09-29T23:28:17.997 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-09-29T23:28:18.117 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-joblib (0.14.0-3) ... 2023-09-29T23:28:18.290 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-09-29T23:28:18.528 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2023-09-29T23:28:18.610 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-29T23:28:18.863 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cachetools (4.0.0-1) ... 2023-09-29T23:28:18.961 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-09-29T23:28:19.139 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2023-09-29T23:28:19.262 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-09-29T23:28:19.432 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2023-09-29T23:28:19.622 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-09-29T23:28:19.766 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-ceph-argparse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:19.915 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-packaging (20.3-1) ... 2023-09-29T23:28:20.055 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2023-09-29T23:28:20.181 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2023-09-29T23:28:20.216 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-09-29T23:28:20.359 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-29T23:28:20.485 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-29T23:28:20.499 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2023-09-29T23:28:20.801 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2023-09-29T23:28:20.811 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-09-29T23:28:20.927 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2023-09-29T23:28:21.213 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-29T23:28:21.328 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-natsort (7.0.1-1) ... 2023-09-29T23:28:21.564 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-29T23:28:21.638 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2023-09-29T23:28:21.916 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up socat (1.7.3.3-2) ... 2023-09-29T23:28:21.939 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2023-09-29T23:28:22.075 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:22.293 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2023-09-29T23:28:22.402 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:28:22.561 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-09-29T23:28:22.602 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-jinja2 (2.10.1-2) ... 2023-09-29T23:28:22.912 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-29T23:28:22.928 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-tempita (0.5.2-6) ... 2023-09-29T23:28:23.039 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-09-29T23:28:23.165 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-09-29T23:28:23.230 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-packaging (20.3-1) ... 2023-09-29T23:28:23.466 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-29T23:28:23.556 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2023-09-29T23:28:23.682 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2023-09-29T23:28:23.784 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:28:23.808 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2023-09-29T23:28:23.935 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-29T23:28:24.061 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-09-29T23:28:24.117 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2023-09-29T23:28:24.388 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-09-29T23:28:24.519 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.1) ... 2023-09-29T23:28:24.699 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-09-29T23:28:24.887 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cryptography (2.8-3ubuntu0.1) ... 2023-09-29T23:28:24.993 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-29T23:28:25.119 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-09-29T23:28:25.298 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up socat (1.7.3.3-2) ... 2023-09-29T23:28:25.420 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-09-29T23:28:25.426 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:25.713 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-29T23:28:25.783 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:28:25.918 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-html5lib (1.0.1-2) ... 2023-09-29T23:28:26.023 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-09-29T23:28:26.269 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2023-09-29T23:28:26.387 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2023-09-29T23:28:26.415 INFO:teuthology.orchestra.run.smithi156.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-09-29T23:28:26.513 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pluggy (0.13.0-2) ... 2023-09-29T23:28:26.695 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-09-29T23:28:26.806 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2023-09-29T23:28:26.846 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-29T23:28:27.124 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:28:27.164 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-09-29T23:28:27.275 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2023-09-29T23:28:27.401 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2023-09-29T23:28:27.524 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-29T23:28:27.650 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-29T23:28:27.737 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2023-09-29T23:28:27.776 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-09-29T23:28:28.021 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-logutils (0.3.3-7) ... 2023-09-29T23:28:28.127 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-29T23:28:28.306 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2023-09-29T23:28:28.412 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up librados2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:28.434 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-09-29T23:28:28.540 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-09-29T23:28:28.711 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-prettytable (0.7.2-5) ... 2023-09-29T23:28:29.003 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2023-09-29T23:28:29.306 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2023-09-29T23:28:29.691 INFO:teuthology.orchestra.run.smithi143.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2023-09-29T23:28:29.966 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2023-09-29T23:28:29.996 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:30.109 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2023-09-29T23:28:30.122 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-29T23:28:30.273 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-09-29T23:28:30.430 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2023-09-29T23:28:30.635 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-09-29T23:28:30.781 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2023-09-29T23:28:30.907 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2023-09-29T23:28:30.961 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-09-29T23:28:31.058 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-webob (1:1.8.5-2) ... 2023-09-29T23:28:31.237 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:28:31.363 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-09-29T23:28:31.385 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2023-09-29T23:28:31.669 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librados2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:31.715 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:31.787 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2023-09-29T23:28:31.840 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-29T23:28:32.167 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:32.293 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-29T23:28:32.636 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-29T23:28:33.229 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libsqlite3-mod-ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:33.355 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2023-09-29T23:28:33.514 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pytest (4.6.9-1) ... 2023-09-29T23:28:33.683 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up librbd1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:33.809 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-29T23:28:33.891 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2023-09-29T23:28:33.935 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-09-29T23:28:34.086 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-09-29T23:28:34.209 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2023-09-29T23:28:34.212 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-rsa (4.0-3) ... 2023-09-29T23:28:34.510 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2023-09-29T23:28:34.538 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-29T23:28:34.636 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-bs4 (4.8.2-1) ... 2023-09-29T23:28:34.664 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-09-29T23:28:34.957 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:34.979 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libcephfs2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:35.105 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2023-09-29T23:28:35.107 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:35.340 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-29T23:28:35.340 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-29T23:28:35.407 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libradosstriper1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:35.525 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2023-09-29T23:28:35.817 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:35.885 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2023-09-29T23:28:36.054 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-29T23:28:36.363 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-09-29T23:28:36.698 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-09-29T23:28:36.916 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librbd1 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:37.025 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2023-09-29T23:28:37.050 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:37.151 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2023-09-29T23:28:37.177 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:37.307 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:37.311 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2023-09-29T23:28:37.437 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rsa (4.0-3) ... 2023-09-29T23:28:37.454 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-routes (2.4.1-2) ... 2023-09-29T23:28:37.724 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2023-09-29T23:28:37.873 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-webtest (2.0.32-2) ... 2023-09-29T23:28:37.898 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-09-29T23:28:38.192 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:38.192 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-modules-core (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:38.308 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:38.341 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:38.463 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-09-29T23:28:38.565 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-29T23:28:38.566 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-09-29T23:28:39.023 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up libcephfs-dev (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:39.248 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2023-09-29T23:28:39.371 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:39.523 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-09-29T23:28:39.849 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2023-09-29T23:28:39.923 INFO:teuthology.orchestra.run.smithi156.stdout:Adding group ceph....done 2023-09-29T23:28:40.185 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rados (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:40.303 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up librgw2 (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:40.359 INFO:teuthology.orchestra.run.smithi156.stdout:Adding system user ceph....done 2023-09-29T23:28:40.429 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rbd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:40.534 INFO:teuthology.orchestra.run.smithi156.stdout:Setting system user ceph properties....done 2023-09-29T23:28:40.558 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up rbd-fuse (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:40.558 INFO:teuthology.orchestra.run.smithi156.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-09-29T23:28:40.700 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-routes (2.4.1-2) ... 2023-09-29T23:28:40.707 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-09-29T23:28:41.001 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-google-auth (1.5.1-2) ... 2023-09-29T23:28:41.023 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-09-29T23:28:41.302 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-rgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:41.428 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-cephfs (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:41.481 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:41.554 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2023-09-29T23:28:41.606 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:41.902 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-29T23:28:41.902 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-29T23:28:42.491 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-common (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:42.533 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:42.953 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-09-29T23:28:43.093 INFO:teuthology.orchestra.run.smithi143.stdout:Adding group ceph....done 2023-09-29T23:28:43.418 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:43.470 INFO:teuthology.orchestra.run.smithi143.stdout:Adding system user ceph....done 2023-09-29T23:28:43.655 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-29T23:28:43.655 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-29T23:28:43.679 INFO:teuthology.orchestra.run.smithi143.stdout:Setting system user ceph properties....done 2023-09-29T23:28:43.701 INFO:teuthology.orchestra.run.smithi143.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-09-29T23:28:43.843 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-09-29T23:28:44.122 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:44.133 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-09-29T23:28:44.357 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-29T23:28:44.357 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-29T23:28:44.583 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-test (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:44.709 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up radosgw (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:44.813 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:44.984 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-29T23:28:44.984 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-09-29T23:28:45.322 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-29T23:28:45.322 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-29T23:28:45.627 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-base (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:45.798 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:45.927 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:46.048 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-09-29T23:28:46.086 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:46.314 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-29T23:28:46.314 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-29T23:28:46.522 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mds (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:46.746 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:46.766 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-29T23:28:46.766 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-09-29T23:28:46.880 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:47.039 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:47.182 INFO:teuthology.orchestra.run.smithi156.stdout:Setting up ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:47.231 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:47.309 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-09-29T23:28:47.460 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-09-29T23:28:47.462 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-29T23:28:47.462 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-09-29T23:28:47.762 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-09-29T23:28:48.216 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-osd (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:48.740 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-29T23:28:48.741 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-09-29T23:28:49.195 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-k8sevents (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:49.321 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-diskprediction-local (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:49.464 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mon (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:49.699 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-29T23:28:49.699 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-09-29T23:28:50.158 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-cephadm (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:50.333 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:50.459 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-mgr-dashboard (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:50.598 INFO:teuthology.orchestra.run.smithi156.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-09-29T23:28:50.601 INFO:teuthology.orchestra.run.smithi143.stdout:Setting up ceph-volume (17.2.6-1165-g93dce5e1-1focal) ... 2023-09-29T23:28:50.753 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2023-09-29T23:28:50.905 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2023-09-29T23:28:51.195 INFO:teuthology.orchestra.run.smithi156.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-29T23:28:51.197 DEBUG:teuthology.parallel:result is None 2023-09-29T23:28:51.214 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for man-db (2.9.1-1) ... 2023-09-29T23:28:54.051 INFO:teuthology.orchestra.run.smithi143.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2023-09-29T23:28:54.692 INFO:teuthology.orchestra.run.smithi143.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-09-29T23:28:54.694 DEBUG:teuthology.parallel:result is None 2023-09-29T23:28:54.695 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-29T23:28:54.841 DEBUG:teuthology.orchestra.run.smithi143:> dpkg-query -W -f '${Version}' ceph 2023-09-29T23:28:54.866 INFO:teuthology.orchestra.run.smithi143.stdout:17.2.6-1165-g93dce5e1-1focal 2023-09-29T23:28:54.866 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1165-g93dce5e1-1focal 2023-09-29T23:28:54.866 INFO:teuthology.task.install:The correct ceph version 17.2.6-1165-g93dce5e1-1focal is installed. 2023-09-29T23:28:54.867 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=93dce5e17918f8aca65993a7a2dc40233a47b481 2023-09-29T23:28:55.091 DEBUG:teuthology.orchestra.run.smithi156:> dpkg-query -W -f '${Version}' ceph 2023-09-29T23:28:55.115 INFO:teuthology.orchestra.run.smithi156.stdout:17.2.6-1165-g93dce5e1-1focal 2023-09-29T23:28:55.115 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1165-g93dce5e1-1focal 2023-09-29T23:28:55.115 INFO:teuthology.task.install:The correct ceph version 17.2.6-1165-g93dce5e1-1focal is installed. 2023-09-29T23:28:55.117 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-29T23:28:55.117 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:28:55.118 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-29T23:28:55.135 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:28:55.135 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-29T23:28:55.172 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-29T23:28:55.173 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:28:55.173 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2023-09-29T23:28:55.192 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-29T23:28:55.203 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:28:55.204 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2023-09-29T23:28:55.219 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-29T23:28:55.276 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-29T23:28:55.277 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:28:55.277 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-29T23:28:55.293 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-29T23:28:55.352 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:28:55.352 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-29T23:28:55.367 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-29T23:28:55.424 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-29T23:28:55.425 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:28:55.425 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2023-09-29T23:28:55.438 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-29T23:28:55.453 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:28:55.453 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2023-09-29T23:28:55.468 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-29T23:28:55.484 INFO:teuthology.run_tasks:Running task ceph... 2023-09-29T23:28:55.568 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-09-29T23:28:55.568 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /var/log/ceph 2023-09-29T23:28:55.571 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /var/log/ceph 2023-09-29T23:28:55.585 INFO:tasks.ceph:Disabling ceph logrotate... 2023-09-29T23:28:55.585 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-29T23:28:55.588 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-29T23:28:55.597 INFO:tasks.ceph:Creating extra log directories... 2023-09-29T23:28:55.597 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-29T23:28:55.599 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-29T23:28:55.662 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-09-29T23:28:55.663 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-09-29T23:28:55.663 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-09-29T23:28:55.664 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:28:55.664 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-09-29T23:28:55.671 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-09-29T23:28:55.707 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:28:55.708 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-09-29T23:28:55.715 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-09-29T23:28:55.799 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-09-29T23:28:55.799 INFO:tasks.ceph:config {'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 3}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 5}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_', 'replacing it with standby', 'No standby daemons available', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-09-29T23:28:55.799 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568', 'branch': 'wip-yuri4-testing-2023-09-29-0928-quincy', 'description': 'rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/failover}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7406568', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 2, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri4-testing-2023-09-29-0928-quincy'}, 'ceph': {'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 3}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 5}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri4-testing-2023-09-29-0928-quincy', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mgr.x', 'mon.a', 'mon.c', 'mds.a', 'mds.c', 'osd.0', 'client.0'], ['mgr.y', 'mgr.z', 'mon.b', 'mds.b', 'osd.1', 'osd.2', 'osd.3', 'client.1']], 'seed': 9810, 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-yuri4-testing-2023-09-29-0928-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481', 'targets': {'smithi143.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM=', 'smithi156.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['overall HEALTH_', '\\(MGR_DOWN\\)', '\\(PG_', 'replacing it with standby', 'No standby daemons available', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'wait-for-scrub': False, 'conf': {'client': {'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'global': {'mon election default strategy': 3}, 'mds': {'debug mgrc': 20}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr ttl cache expire seconds': 5}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'sha1': '93dce5e17918f8aca65993a7a2dc40233a47b481', 'cluster': 'ceph'}}, {'cephfs_test_runner': {'modules': ['tasks.mgr.test_failover']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-09-29_19:38:56', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-09-29T23:28:55.800 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-29T23:28:55.802 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-29T23:28:55.808 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m0777 -- /var/run/ceph 2023-09-29T23:28:55.810 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/run/ceph 2023-09-29T23:28:55.824 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:28:55.824 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2023-09-29T23:28:55.871 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-29T23:28:55.872 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2023-09-29T23:28:55.878 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-29T23:28:55.878 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T23:28:55.878 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 358 Links: 1 2023-09-29T23:28:55.878 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T23:28:55.878 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-09-29 23:25:15.047219864 +0000 2023-09-29T23:28:55.878 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-09-29 23:25:15.043219955 +0000 2023-09-29T23:28:55.878 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-09-29 23:25:15.043219955 +0000 2023-09-29T23:28:55.878 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-09-29T23:28:55.879 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-29T23:28:55.893 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-09-29T23:28:55.894 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-09-29T23:28:55.894 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000366713 s, 1.4 MB/s 2023-09-29T23:28:55.895 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-29T23:28:55.903 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2023-09-29T23:28:55.910 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-29T23:28:55.910 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T23:28:55.910 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 366 Links: 1 2023-09-29T23:28:55.910 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T23:28:55.911 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-09-29 23:25:15.815202195 +0000 2023-09-29T23:28:55.911 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-09-29 23:25:15.531208728 +0000 2023-09-29T23:28:55.911 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-09-29 23:25:15.531208728 +0000 2023-09-29T23:28:55.911 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-09-29T23:28:55.911 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-29T23:28:55.924 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-09-29T23:28:55.924 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-09-29T23:28:55.924 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000350035 s, 1.5 MB/s 2023-09-29T23:28:55.925 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-29T23:28:55.933 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2023-09-29T23:28:55.987 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-29T23:28:55.988 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T23:28:55.988 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 371 Links: 1 2023-09-29T23:28:55.988 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T23:28:55.988 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-09-29 23:25:16.307190875 +0000 2023-09-29T23:28:55.988 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-09-29 23:25:16.023197409 +0000 2023-09-29T23:28:55.988 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-09-29 23:25:16.023197409 +0000 2023-09-29T23:28:55.988 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-09-29T23:28:55.988 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-29T23:28:55.999 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-09-29T23:28:55.999 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-09-29T23:28:55.999 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000417651 s, 1.2 MB/s 2023-09-29T23:28:56.000 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-29T23:28:56.007 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2023-09-29T23:28:56.055 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-29T23:28:56.056 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T23:28:56.056 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 6h/6d Inode: 379 Links: 1 2023-09-29T23:28:56.056 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T23:28:56.056 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2023-09-29 23:25:16.839178635 +0000 2023-09-29T23:28:56.056 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2023-09-29 23:25:16.555185169 +0000 2023-09-29T23:28:56.056 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2023-09-29 23:25:16.555185169 +0000 2023-09-29T23:28:56.056 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: - 2023-09-29T23:28:56.056 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-29T23:28:56.068 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2023-09-29T23:28:56.069 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2023-09-29T23:28:56.069 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000407841 s, 1.3 MB/s 2023-09-29T23:28:56.070 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-29T23:28:56.078 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2023-09-29T23:28:56.078 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:28:56.078 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2023-09-29T23:28:56.085 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-29T23:28:56.085 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2023-09-29T23:28:56.092 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-29T23:28:56.092 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T23:28:56.092 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 361 Links: 1 2023-09-29T23:28:56.092 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T23:28:56.092 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-29 23:25:15.337295231 +0000 2023-09-29T23:28:56.092 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-29 23:25:15.045301962 +0000 2023-09-29T23:28:56.093 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-29 23:25:15.045301962 +0000 2023-09-29T23:28:56.093 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-29T23:28:56.093 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-29T23:28:56.104 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-29T23:28:56.104 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-29T23:28:56.104 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000299449 s, 1.7 MB/s 2023-09-29T23:28:56.106 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-29T23:28:56.162 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2023-09-29T23:28:56.169 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-29T23:28:56.169 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T23:28:56.169 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 367 Links: 1 2023-09-29T23:28:56.169 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T23:28:56.169 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-29 23:25:15.837283706 +0000 2023-09-29T23:28:56.170 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-29 23:25:15.533290713 +0000 2023-09-29T23:28:56.170 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-29 23:25:15.533290713 +0000 2023-09-29T23:28:56.170 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-29T23:28:56.170 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-29T23:28:56.182 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-29T23:28:56.182 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-29T23:28:56.182 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000212898 s, 2.4 MB/s 2023-09-29T23:28:56.183 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-29T23:28:56.191 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2023-09-29T23:28:56.195 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-29T23:28:56.196 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T23:28:56.196 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 373 Links: 1 2023-09-29T23:28:56.196 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T23:28:56.196 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-29 23:25:16.313272734 +0000 2023-09-29T23:28:56.196 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-29 23:25:16.041279004 +0000 2023-09-29T23:28:56.196 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-29 23:25:16.041279004 +0000 2023-09-29T23:28:56.196 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-29T23:28:56.196 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-29T23:28:56.251 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-29T23:28:56.251 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-29T23:28:56.251 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000290964 s, 1.8 MB/s 2023-09-29T23:28:56.252 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-29T23:28:56.259 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2023-09-29T23:28:56.265 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-29T23:28:56.265 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-29T23:28:56.265 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 6h/6d Inode: 378 Links: 1 2023-09-29T23:28:56.265 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-29T23:28:56.265 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-09-29 23:25:16.773262131 +0000 2023-09-29T23:28:56.265 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-09-29 23:25:16.513268124 +0000 2023-09-29T23:28:56.265 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-09-29 23:25:16.513268124 +0000 2023-09-29T23:28:56.265 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: - 2023-09-29T23:28:56.266 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-29T23:28:56.276 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-09-29T23:28:56.276 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-09-29T23:28:56.276 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.00028788 s, 1.8 MB/s 2023-09-29T23:28:56.277 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-29T23:28:56.329 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2023-09-29T23:28:56.329 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2023-09-29T23:28:56.329 INFO:tasks.ceph:Generating config... 2023-09-29T23:28:56.331 INFO:tasks.ceph:[client] debug client = 20 2023-09-29T23:28:56.331 INFO:tasks.ceph:[client] debug mgrc = 20 2023-09-29T23:28:56.331 INFO:tasks.ceph:[client] debug ms = 1 2023-09-29T23:28:56.331 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-09-29T23:28:56.331 INFO:tasks.ceph:[mds] debug mgrc = 20 2023-09-29T23:28:56.331 INFO:tasks.ceph:[mgr] debug client = 20 2023-09-29T23:28:56.331 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-09-29T23:28:56.331 INFO:tasks.ceph:[mgr] debug ms = 1 2023-09-29T23:28:56.331 INFO:tasks.ceph:[mgr] mgr ttl cache expire seconds = 5 2023-09-29T23:28:56.331 INFO:tasks.ceph:[mon] debug mon = 20 2023-09-29T23:28:56.332 INFO:tasks.ceph:[mon] debug ms = 1 2023-09-29T23:28:56.332 INFO:tasks.ceph:[mon] debug paxos = 20 2023-09-29T23:28:56.332 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-09-29T23:28:56.332 INFO:tasks.ceph:[osd] bluestore compression algorithm = snappy 2023-09-29T23:28:56.332 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2023-09-29T23:28:56.332 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-09-29T23:28:56.332 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-09-29T23:28:56.332 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-09-29T23:28:56.332 INFO:tasks.ceph:[osd] debug mgrc = 20 2023-09-29T23:28:56.332 INFO:tasks.ceph:[osd] debug ms = 1 2023-09-29T23:28:56.332 INFO:tasks.ceph:[osd] debug osd = 20 2023-09-29T23:28:56.332 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-09-29T23:28:56.332 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-09-29T23:28:56.333 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-09-29T23:28:56.333 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-09-29T23:28:56.333 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-09-29T23:28:56.333 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-09-29T23:28:56.333 INFO:tasks.ceph:Setting up mon.a... 2023-09-29T23:28:56.333 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-09-29T23:28:56.369 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.keyring 2023-09-29T23:28:56.373 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-09-29T23:28:56.455 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-29T23:28:56.468 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.143'), ('mon.c', '[v2:172.21.15.143:3301,v1:172.21.15.143:6790]'), ('mon.b', '172.21.15.156')] 2023-09-29T23:28:56.468 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.143,[v2:172.21.15.143:3301,v1:172.21.15.143:6790],172.21.15.156', 'mon election default strategy': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'snappy', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug mgrc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'mgr ttl cache expire seconds': 5}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug client': 20, 'debug mgrc': 20, 'debug ms': 1}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds': {'debug mgrc': 20}} 2023-09-29T23:28:56.469 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:28:56.469 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-09-29T23:28:56.473 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.143 --addv c '[v2:172.21.15.143:3301,v1:172.21.15.143:6790]' --add b 172.21.15.156 --print /home/ubuntu/cephtest/ceph.monmap 2023-09-29T23:28:56.508 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-09-29T23:28:56.508 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: generated fsid d0aefd66-f5df-4d76-82e3-362118c32b1d 2023-09-29T23:28:56.508 INFO:teuthology.orchestra.run.smithi143.stdout:setting min_mon_release = octopus 2023-09-29T23:28:56.509 INFO:teuthology.orchestra.run.smithi143.stdout:epoch 0 2023-09-29T23:28:56.509 INFO:teuthology.orchestra.run.smithi143.stdout:fsid d0aefd66-f5df-4d76-82e3-362118c32b1d 2023-09-29T23:28:56.509 INFO:teuthology.orchestra.run.smithi143.stdout:last_changed 2023-09-29T23:28:56.508983+0000 2023-09-29T23:28:56.509 INFO:teuthology.orchestra.run.smithi143.stdout:created 2023-09-29T23:28:56.508983+0000 2023-09-29T23:28:56.509 INFO:teuthology.orchestra.run.smithi143.stdout:min_mon_release 15 (octopus) 2023-09-29T23:28:56.509 INFO:teuthology.orchestra.run.smithi143.stdout:election_strategy: 3 2023-09-29T23:28:56.509 INFO:teuthology.orchestra.run.smithi143.stdout:0: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.a 2023-09-29T23:28:56.509 INFO:teuthology.orchestra.run.smithi143.stdout:1: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.b 2023-09-29T23:28:56.509 INFO:teuthology.orchestra.run.smithi143.stdout:2: [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] mon.c 2023-09-29T23:28:56.509 INFO:teuthology.orchestra.run.smithi143.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-09-29T23:28:56.510 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-09-29T23:28:56.514 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d0aefd66-f5df-4d76-82e3-362118c32b1d... 2023-09-29T23:28:56.515 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-29T23:28:56.561 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-29T23:28:56.595 INFO:teuthology.orchestra.run.smithi143.stdout:[global] 2023-09-29T23:28:56.596 INFO:teuthology.orchestra.run.smithi143.stdout: chdir = "" 2023-09-29T23:28:56.596 INFO:teuthology.orchestra.run.smithi143.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-29T23:28:56.596 INFO:teuthology.orchestra.run.smithi143.stdout: auth supported = cephx 2023-09-29T23:28:56.596 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.596 INFO:teuthology.orchestra.run.smithi143.stdout: filestore xattr use omap = true 2023-09-29T23:28:56.596 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.596 INFO:teuthology.orchestra.run.smithi143.stdout: mon clock drift allowed = 1.000 2023-09-29T23:28:56.596 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.596 INFO:teuthology.orchestra.run.smithi143.stdout: osd crush chooseleaf type = 0 2023-09-29T23:28:56.596 INFO:teuthology.orchestra.run.smithi143.stdout: auth debug = true 2023-09-29T23:28:56.597 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.597 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on old message = true 2023-09-29T23:28:56.597 INFO:teuthology.orchestra.run.smithi143.stdout: ms die on bug = true 2023-09-29T23:28:56.597 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.597 INFO:teuthology.orchestra.run.smithi143.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-29T23:28:56.597 INFO:teuthology.orchestra.run.smithi143.stdout: mon pg warn max object skew = 0 2023-09-29T23:28:56.597 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.597 INFO:teuthology.orchestra.run.smithi143.stdout: # disable pg_autoscaler by default for new pools 2023-09-29T23:28:56.597 INFO:teuthology.orchestra.run.smithi143.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-29T23:28:56.597 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.598 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default size = 2 2023-09-29T23:28:56.598 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.598 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow primary affinity = true 2023-09-29T23:28:56.598 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd allow pg remap = true 2023-09-29T23:28:56.598 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on legacy crush tunables = false 2023-09-29T23:28:56.598 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on crush straw calc version zero = false 2023-09-29T23:28:56.598 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on no sortbitwise = false 2023-09-29T23:28:56.599 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on osd down out interval zero = false 2023-09-29T23:28:56.599 INFO:teuthology.orchestra.run.smithi143.stdout: mon warn on too few osds = false 2023-09-29T23:28:56.599 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-29T23:28:56.599 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-29T23:28:56.599 INFO:teuthology.orchestra.run.smithi143.stdout: mon_allow_pool_size_one = true 2023-09-29T23:28:56.599 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.599 INFO:teuthology.orchestra.run.smithi143.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-29T23:28:56.599 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.599 INFO:teuthology.orchestra.run.smithi143.stdout: osd default data pool replay window = 5 2023-09-29T23:28:56.600 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.600 INFO:teuthology.orchestra.run.smithi143.stdout: mon allow pool delete = true 2023-09-29T23:28:56.600 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.600 INFO:teuthology.orchestra.run.smithi143.stdout: mon cluster log file level = debug 2023-09-29T23:28:56.600 INFO:teuthology.orchestra.run.smithi143.stdout: debug asserts on shutdown = true 2023-09-29T23:28:56.600 INFO:teuthology.orchestra.run.smithi143.stdout: mon health detail to clog = false 2023-09-29T23:28:56.600 INFO:teuthology.orchestra.run.smithi143.stdout: mon host = "172.21.15.143,[v2:172.21.15.143:3301,v1:172.21.15.143:6790],172.21.15.156" 2023-09-29T23:28:56.600 INFO:teuthology.orchestra.run.smithi143.stdout: mon election default strategy = 3 2023-09-29T23:28:56.600 INFO:teuthology.orchestra.run.smithi143.stdout: fsid = d0aefd66-f5df-4d76-82e3-362118c32b1d 2023-09-29T23:28:56.600 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.600 INFO:teuthology.orchestra.run.smithi143.stdout:[osd] 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: osd journal size = 100 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub load threshold = 5.0 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: osd scrub max interval = 600 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: osd mclock profile = high_recovery_ops 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: osd recover clone overlap = true 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: osd recovery max chunk = 1048576 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug shutdown = true 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug op order = true 2023-09-29T23:28:56.601 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug verify stray on activate = true 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: osd open classes on start = true 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug pg log writeout = true 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: osd deep scrub update digest min age = 30 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: osd map max advance = 10 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: journal zero on create = true 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: filestore ondisk finisher threads = 3 2023-09-29T23:28:56.602 INFO:teuthology.orchestra.run.smithi143.stdout: filestore apply finisher threads = 3 2023-09-29T23:28:56.603 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.603 INFO:teuthology.orchestra.run.smithi143.stdout: bdev debug aio = true 2023-09-29T23:28:56.603 INFO:teuthology.orchestra.run.smithi143.stdout: osd debug misdirected ops = true 2023-09-29T23:28:56.603 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore block size = 96636764160 2023-09-29T23:28:56.603 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore compression algorithm = snappy 2023-09-29T23:28:56.603 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore compression mode = aggressive 2023-09-29T23:28:56.603 INFO:teuthology.orchestra.run.smithi143.stdout: bluestore fsck on mount = True 2023-09-29T23:28:56.603 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluefs = 20 2023-09-29T23:28:56.603 INFO:teuthology.orchestra.run.smithi143.stdout: debug bluestore = 20 2023-09-29T23:28:56.603 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgrc = 20 2023-09-29T23:28:56.603 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout: debug osd = 20 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout: debug rocksdb = 10 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd backfillfull_ratio = 0.85 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd full ratio = 0.9 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd nearfull ratio = 0.8 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout: osd failsafe full ratio = 0.95 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout: osd objectstore = bluestore 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr] 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgr = 20 2023-09-29T23:28:56.604 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-09-29T23:28:56.605 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-09-29T23:28:56.605 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-09-29T23:28:56.605 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-09-29T23:28:56.605 INFO:teuthology.orchestra.run.smithi143.stdout: mgr/telemetry/nag = false 2023-09-29T23:28:56.605 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2023-09-29T23:28:56.605 INFO:teuthology.orchestra.run.smithi143.stdout: mgr ttl cache expire seconds = 5 2023-09-29T23:28:56.605 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.605 INFO:teuthology.orchestra.run.smithi143.stdout:[mon] 2023-09-29T23:28:56.605 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-09-29T23:28:56.605 INFO:teuthology.orchestra.run.smithi143.stdout: debug mon = 20 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: debug paxos = 20 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: debug auth = 20 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: mon data avail warn = 5 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: mon mgr mkfs grace = 240 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min pgs per osd = 4 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd reporter subtree level = osd 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: mon osd prime pg temp = true 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: mon reweight min bytes per osd = 10 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: auth mon ticket ttl = 660 # 11m 2023-09-29T23:28:56.606 INFO:teuthology.orchestra.run.smithi143.stdout: auth service ticket ttl = 240 # 4m 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout: # don't complain about insecure global_id in the test suite 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout: # 1m isn't quite enough 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout: mon_down_mkfs_grace = 2m 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout: mon_warn_on_filestore_osds = false 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout:[client] 2023-09-29T23:28:56.607 INFO:teuthology.orchestra.run.smithi143.stdout: rgw cache enabled = true 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable ops log = true 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout: rgw enable usage log = true 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout: debug client = 20 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgrc = 20 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout: debug ms = 1 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.a] 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.c] 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout:[mon.b] 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2023-09-29T23:28:56.608 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2023-09-29T23:28:56.609 INFO:teuthology.orchestra.run.smithi143.stdout:[mds] 2023-09-29T23:28:56.609 INFO:teuthology.orchestra.run.smithi143.stdout: debug mgrc = 20 2023-09-29T23:28:56.610 INFO:teuthology.orchestra.run.smithi156.stdout:[global] 2023-09-29T23:28:56.610 INFO:teuthology.orchestra.run.smithi156.stdout: chdir = "" 2023-09-29T23:28:56.610 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-29T23:28:56.611 INFO:teuthology.orchestra.run.smithi156.stdout: auth supported = cephx 2023-09-29T23:28:56.611 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.611 INFO:teuthology.orchestra.run.smithi156.stdout: filestore xattr use omap = true 2023-09-29T23:28:56.611 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.611 INFO:teuthology.orchestra.run.smithi156.stdout: mon clock drift allowed = 1.000 2023-09-29T23:28:56.611 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.611 INFO:teuthology.orchestra.run.smithi156.stdout: osd crush chooseleaf type = 0 2023-09-29T23:28:56.611 INFO:teuthology.orchestra.run.smithi156.stdout: auth debug = true 2023-09-29T23:28:56.611 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.611 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on old message = true 2023-09-29T23:28:56.612 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on bug = true 2023-09-29T23:28:56.612 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.612 INFO:teuthology.orchestra.run.smithi156.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-29T23:28:56.612 INFO:teuthology.orchestra.run.smithi156.stdout: mon pg warn max object skew = 0 2023-09-29T23:28:56.612 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.612 INFO:teuthology.orchestra.run.smithi156.stdout: # disable pg_autoscaler by default for new pools 2023-09-29T23:28:56.612 INFO:teuthology.orchestra.run.smithi156.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-29T23:28:56.612 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.612 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default size = 2 2023-09-29T23:28:56.612 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.612 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow primary affinity = true 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow pg remap = true 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on legacy crush tunables = false 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on crush straw calc version zero = false 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on no sortbitwise = false 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on osd down out interval zero = false 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on too few osds = false 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: mon_allow_pool_size_one = true 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-29T23:28:56.613 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.614 INFO:teuthology.orchestra.run.smithi156.stdout: osd default data pool replay window = 5 2023-09-29T23:28:56.614 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.614 INFO:teuthology.orchestra.run.smithi156.stdout: mon allow pool delete = true 2023-09-29T23:28:56.614 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.614 INFO:teuthology.orchestra.run.smithi156.stdout: mon cluster log file level = debug 2023-09-29T23:28:56.614 INFO:teuthology.orchestra.run.smithi156.stdout: debug asserts on shutdown = true 2023-09-29T23:28:56.614 INFO:teuthology.orchestra.run.smithi156.stdout: mon health detail to clog = false 2023-09-29T23:28:56.614 INFO:teuthology.orchestra.run.smithi156.stdout: mon host = "172.21.15.143,[v2:172.21.15.143:3301,v1:172.21.15.143:6790],172.21.15.156" 2023-09-29T23:28:56.614 INFO:teuthology.orchestra.run.smithi156.stdout: mon election default strategy = 3 2023-09-29T23:28:56.614 INFO:teuthology.orchestra.run.smithi156.stdout: fsid = d0aefd66-f5df-4d76-82e3-362118c32b1d 2023-09-29T23:28:56.615 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.615 INFO:teuthology.orchestra.run.smithi156.stdout:[osd] 2023-09-29T23:28:56.615 INFO:teuthology.orchestra.run.smithi156.stdout: osd journal size = 100 2023-09-29T23:28:56.615 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.615 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub load threshold = 5.0 2023-09-29T23:28:56.615 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub max interval = 600 2023-09-29T23:28:56.615 INFO:teuthology.orchestra.run.smithi156.stdout: osd mclock profile = high_recovery_ops 2023-09-29T23:28:56.615 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.615 INFO:teuthology.orchestra.run.smithi156.stdout: osd recover clone overlap = true 2023-09-29T23:28:56.615 INFO:teuthology.orchestra.run.smithi156.stdout: osd recovery max chunk = 1048576 2023-09-29T23:28:56.615 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug shutdown = true 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug op order = true 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug verify stray on activate = true 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: osd open classes on start = true 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug pg log writeout = true 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: osd deep scrub update digest min age = 30 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: osd map max advance = 10 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.616 INFO:teuthology.orchestra.run.smithi156.stdout: journal zero on create = true 2023-09-29T23:28:56.617 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.617 INFO:teuthology.orchestra.run.smithi156.stdout: filestore ondisk finisher threads = 3 2023-09-29T23:28:56.617 INFO:teuthology.orchestra.run.smithi156.stdout: filestore apply finisher threads = 3 2023-09-29T23:28:56.617 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.617 INFO:teuthology.orchestra.run.smithi156.stdout: bdev debug aio = true 2023-09-29T23:28:56.617 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug misdirected ops = true 2023-09-29T23:28:56.617 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore block size = 96636764160 2023-09-29T23:28:56.617 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore compression algorithm = snappy 2023-09-29T23:28:56.617 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore compression mode = aggressive 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore fsck on mount = True 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluefs = 20 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluestore = 20 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgrc = 20 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: debug osd = 20 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: debug rocksdb = 10 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd backfillfull_ratio = 0.85 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd full ratio = 0.9 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd nearfull ratio = 0.8 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: osd failsafe full ratio = 0.95 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: osd objectstore = bluestore 2023-09-29T23:28:56.618 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr] 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgr = 20 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout: mgr/telemetry/nag = false 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout: mgr ttl cache expire seconds = 5 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.619 INFO:teuthology.orchestra.run.smithi156.stdout:[mon] 2023-09-29T23:28:56.620 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-09-29T23:28:56.620 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2023-09-29T23:28:56.620 INFO:teuthology.orchestra.run.smithi156.stdout: debug paxos = 20 2023-09-29T23:28:56.620 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2023-09-29T23:28:56.620 INFO:teuthology.orchestra.run.smithi156.stdout: mon data avail warn = 5 2023-09-29T23:28:56.620 INFO:teuthology.orchestra.run.smithi156.stdout: mon mgr mkfs grace = 240 2023-09-29T23:28:56.620 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2023-09-29T23:28:56.620 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd reporter subtree level = osd 2023-09-29T23:28:56.620 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd prime pg temp = true 2023-09-29T23:28:56.620 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2023-09-29T23:28:56.620 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: auth mon ticket ttl = 660 # 11m 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: auth service ticket ttl = 240 # 4m 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: # don't complain about insecure global_id in the test suite 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: # 1m isn't quite enough 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: mon_down_mkfs_grace = 2m 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.621 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_filestore_osds = false 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout:[client] 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout: rgw cache enabled = true 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable ops log = true 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable usage log = true 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgrc = 20 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.a] 2023-09-29T23:28:56.622 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.c] 2023-09-29T23:28:56.623 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.b] 2023-09-29T23:28:56.623 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2023-09-29T23:28:56.623 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2023-09-29T23:28:56.623 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2023-09-29T23:28:56.623 INFO:teuthology.orchestra.run.smithi156.stdout:[mds] 2023-09-29T23:28:56.623 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgrc = 20 2023-09-29T23:28:56.623 INFO:tasks.ceph:Creating admin key on mon.a... 2023-09-29T23:28:56.623 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-09-29T23:28:56.672 INFO:tasks.ceph:Copying monmap to all nodes... 2023-09-29T23:28:56.672 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:28:56.672 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-09-29T23:28:56.719 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:28:56.720 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-09-29T23:28:56.727 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T23:28:56.727 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:28:56.727 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-29T23:28:56.727 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-29T23:28:56.754 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:28:56.754 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-29T23:28:56.803 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi156.front.sepia.ceph.com 2023-09-29T23:28:56.803 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:28:56.803 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-29T23:28:56.804 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-29T23:28:56.830 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:28:56.830 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-29T23:28:56.883 INFO:tasks.ceph:Setting up mon nodes... 2023-09-29T23:28:56.883 INFO:tasks.ceph:Setting up mgr nodes... 2023-09-29T23:28:56.884 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-09-29T23:28:56.928 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-09-29T23:28:56.930 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-09-29T23:28:56.975 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-09-29T23:28:56.978 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mgr/ceph-z && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.z /var/lib/ceph/mgr/ceph-z/keyring 2023-09-29T23:28:57.020 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mgr/ceph-z/keyring 2023-09-29T23:28:57.022 INFO:tasks.ceph:Setting up mds nodes... 2023-09-29T23:28:57.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-09-29T23:28:57.071 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-09-29T23:28:57.073 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-09-29T23:28:57.087 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-09-29T23:28:57.126 INFO:teuthology.orchestra.run.smithi143.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-09-29T23:28:57.129 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-09-29T23:28:57.144 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-09-29T23:28:57.188 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-09-29T23:28:57.190 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-09-29T23:28:57.201 INFO:tasks.ceph_client:Setting up client nodes... 2023-09-29T23:28:57.201 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-29T23:28:57.238 INFO:teuthology.orchestra.run.smithi143.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-09-29T23:28:57.251 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-09-29T23:28:57.287 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-09-29T23:28:57.301 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-09-29T23:28:57.301 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi143.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2023-09-29T23:28:57.301 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-09-29T23:28:57.317 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2023-09-29T23:28:57.317 INFO:tasks.ceph:role: osd.0 2023-09-29T23:28:57.317 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T23:28:57.317 DEBUG:teuthology.orchestra.run.smithi143:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-29T23:28:57.824 INFO:teuthology.orchestra.run.smithi143.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-29T23:28:57.825 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-29T23:28:57.825 INFO:teuthology.orchestra.run.smithi143.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-29T23:28:57.825 INFO:teuthology.orchestra.run.smithi143.stdout: = reflink=1 2023-09-29T23:28:57.825 INFO:teuthology.orchestra.run.smithi143.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-29T23:28:57.825 INFO:teuthology.orchestra.run.smithi143.stdout: = sunit=0 swidth=0 blks 2023-09-29T23:28:57.825 INFO:teuthology.orchestra.run.smithi143.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-29T23:28:57.825 INFO:teuthology.orchestra.run.smithi143.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-29T23:28:57.825 INFO:teuthology.orchestra.run.smithi143.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-29T23:28:57.825 INFO:teuthology.orchestra.run.smithi143.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-29T23:28:57.827 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi143.front.sepia.ceph.com -o noatime 2023-09-29T23:28:57.827 DEBUG:teuthology.orchestra.run.smithi143:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-09-29T23:28:57.896 DEBUG:teuthology.orchestra.run.smithi143:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-09-29T23:28:57.908 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: /sbin/restorecon: command not found 2023-09-29T23:28:57.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T23:28:57.910 DEBUG:teuthology.orchestra.run.smithi143:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-29T23:28:58.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:28:58.041+0000 7f3819c613c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-09-29T23:28:58.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:28:58.041+0000 7f3819c613c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-09-29T23:28:58.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:28:58.041+0000 7f3819c613c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-29T23:28:58.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:28:58.041+0000 7f3819c613c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-29T23:28:58.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:28:58.041+0000 7f3819c613c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-29T23:28:58.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:28:58.053+0000 7f3819c613c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-09-29T23:28:58.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:28:58.609+0000 7f3819c613c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:28:59.393 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:28:59.393+0000 7f3819c613c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:29:00.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:00.177+0000 7f3819c613c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:29:01.031 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-09-29T23:29:01.046 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-09-29T23:29:01.062 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2023-09-29T23:29:01.062 INFO:tasks.ceph:role: osd.1 2023-09-29T23:29:01.062 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com 2023-09-29T23:29:01.062 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-29T23:29:01.104 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-29T23:29:01.104 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-29T23:29:01.104 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-29T23:29:01.104 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 2023-09-29T23:29:01.104 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-29T23:29:01.105 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-09-29T23:29:01.105 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-29T23:29:01.105 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-29T23:29:01.105 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-29T23:29:01.105 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-29T23:29:01.106 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-09-29T23:29:01.106 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2023-09-29T23:29:01.125 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-09-29T23:29:01.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T23:29:01.133 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: /sbin/restorecon: command not found 2023-09-29T23:29:01.133 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-09-29T23:29:01.193 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2023-09-29T23:29:01.193 INFO:tasks.ceph:role: osd.2 2023-09-29T23:29:01.194 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com 2023-09-29T23:29:01.194 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-29T23:29:01.236 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-29T23:29:01.236 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-29T23:29:01.236 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-29T23:29:01.236 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 2023-09-29T23:29:01.236 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-29T23:29:01.236 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-09-29T23:29:01.237 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-29T23:29:01.237 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-29T23:29:01.237 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-29T23:29:01.237 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-29T23:29:01.238 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-09-29T23:29:01.238 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2023-09-29T23:29:01.259 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-09-29T23:29:01.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T23:29:01.312 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: /sbin/restorecon: command not found 2023-09-29T23:29:01.312 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-09-29T23:29:01.323 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2023-09-29T23:29:01.323 INFO:tasks.ceph:role: osd.3 2023-09-29T23:29:01.323 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com 2023-09-29T23:29:01.323 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-29T23:29:01.365 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-29T23:29:01.365 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-29T23:29:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-29T23:29:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 2023-09-29T23:29:01.366 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-29T23:29:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-09-29T23:29:01.366 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-29T23:29:01.366 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-29T23:29:01.366 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-29T23:29:01.366 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-29T23:29:01.367 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-09-29T23:29:01.367 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2023-09-29T23:29:01.387 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-09-29T23:29:01.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T23:29:01.397 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: /sbin/restorecon: command not found 2023-09-29T23:29:01.397 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-29T23:29:01.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:01.457+0000 7fc2c11bf3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-09-29T23:29:01.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:01.457+0000 7fc2c11bf3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-09-29T23:29:01.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:01.457+0000 7fc2c11bf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-29T23:29:01.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:01.457+0000 7fc2c11bf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-29T23:29:01.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:01.457+0000 7fc2c11bf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-29T23:29:01.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:01.457+0000 7fc2c11bf3c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-09-29T23:29:02.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:02.009+0000 7fc2c11bf3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:29:02.793 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:02.789+0000 7fc2c11bf3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:29:03.582 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:03.581+0000 7fc2c11bf3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:29:04.434 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-09-29T23:29:04.447 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-29T23:29:04.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:04.485+0000 7f7ab57983c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-09-29T23:29:04.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:04.485+0000 7f7ab57983c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-09-29T23:29:04.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:04.485+0000 7f7ab57983c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-29T23:29:04.486 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:04.485+0000 7f7ab57983c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-29T23:29:04.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:04.485+0000 7f7ab57983c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-29T23:29:04.487 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:04.485+0000 7f7ab57983c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-09-29T23:29:05.040 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:05.037+0000 7f7ab57983c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:29:05.822 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:05.821+0000 7f7ab57983c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:29:06.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:06.601+0000 7f7ab57983c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:29:07.458 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-09-29T23:29:07.475 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-29T23:29:07.520 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:07.517+0000 7f96eb60f3c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-09-29T23:29:07.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:07.517+0000 7f96eb60f3c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-09-29T23:29:07.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:07.517+0000 7f96eb60f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-29T23:29:07.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:07.517+0000 7f96eb60f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-29T23:29:07.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:07.517+0000 7f96eb60f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-29T23:29:07.521 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:07.517+0000 7f96eb60f3c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-09-29T23:29:08.084 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:08.081+0000 7f96eb60f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:29:08.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:08.857+0000 7f96eb60f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:29:09.638 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:09.637+0000 7f96eb60f3c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-09-29T23:29:10.501 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-09-29T23:29:10.516 INFO:tasks.ceph:Reading keys from all nodes... 2023-09-29T23:29:10.516 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:29:10.516 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-09-29T23:29:10.532 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:29:10.533 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-09-29T23:29:10.549 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:29:10.549 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-09-29T23:29:10.611 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:29:10.611 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-09-29T23:29:10.620 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:29:10.620 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-09-29T23:29:10.636 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:29:10.636 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mgr/ceph-z/keyring of=/dev/stdout 2023-09-29T23:29:10.692 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:29:10.693 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-09-29T23:29:10.703 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:29:10.703 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-09-29T23:29:10.717 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:29:10.717 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-09-29T23:29:10.776 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:29:10.776 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-09-29T23:29:10.786 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:29:10.786 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-09-29T23:29:10.793 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:29:10.793 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-09-29T23:29:10.798 INFO:tasks.ceph:Adding keys to all mons... 2023-09-29T23:29:10.798 DEBUG:teuthology.orchestra.run.smithi143:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-29T23:29:10.838 DEBUG:teuthology.orchestra.run.smithi156:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-29T23:29:10.851 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.x] 2023-09-29T23:29:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC4XRdlKEBgNxAAbUWSN+woXyZQ8vJ+S8C5HA== 2023-09-29T23:29:10.851 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.a] 2023-09-29T23:29:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC5XRdlLC9GBBAACB/7DMcll4Y2ohaOhfvciw== 2023-09-29T23:29:10.851 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.c] 2023-09-29T23:29:10.851 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC5XRdlPbOUBxAAC2tRt7JzL++zJTpuWHozpQ== 2023-09-29T23:29:10.851 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.0] 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC6XRdlYiiCAhAAWcKL+KO19c3Ibs7cdQX9lw== 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.y] 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC4XRdleFQsOhAAsS+lCjIx+YVeqx67suXCxw== 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout:[mgr.z] 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC5XRdlgvA6ARAAgfXzVXokGHGs/MKSdp8HVQ== 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout:[mds.b] 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC5XRdlBMI+CxAAiPZNWrDxspI9MBLYRnQYaw== 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.1] 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC9XRdlqNNZGxAAUuQj7nDt7Mg2oqSETvgw7A== 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.2] 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDAXRdlBNr9HBAAPGa9EXA40S7bwfTHmlU4TQ== 2023-09-29T23:29:10.852 INFO:teuthology.orchestra.run.smithi143.stdout:[osd.3] 2023-09-29T23:29:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQDDXRdl8uIJHxAArpHRi4pHW12ga1CnmcSP7g== 2023-09-29T23:29:10.853 INFO:teuthology.orchestra.run.smithi143.stdout:[client.0] 2023-09-29T23:29:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC5XRdlLUE4DhAAwloMjGPQNpmB+ZmcCIQThA== 2023-09-29T23:29:10.853 INFO:teuthology.orchestra.run.smithi143.stdout:[client.1] 2023-09-29T23:29:10.853 INFO:teuthology.orchestra.run.smithi143.stdout: key = AQC5XRdlflwmERAAa4kjNyBs6UVlfO4T8LhfTQ== 2023-09-29T23:29:10.855 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.x] 2023-09-29T23:29:10.855 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQC4XRdlKEBgNxAAbUWSN+woXyZQ8vJ+S8C5HA== 2023-09-29T23:29:10.855 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2023-09-29T23:29:10.855 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQC5XRdlLC9GBBAACB/7DMcll4Y2ohaOhfvciw== 2023-09-29T23:29:10.855 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2023-09-29T23:29:10.855 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQC5XRdlPbOUBxAAC2tRt7JzL++zJTpuWHozpQ== 2023-09-29T23:29:10.855 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.0] 2023-09-29T23:29:10.855 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQC6XRdlYiiCAhAAWcKL+KO19c3Ibs7cdQX9lw== 2023-09-29T23:29:10.855 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.y] 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQC4XRdleFQsOhAAsS+lCjIx+YVeqx67suXCxw== 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.z] 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQC5XRdlgvA6ARAAgfXzVXokGHGs/MKSdp8HVQ== 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQC5XRdlBMI+CxAAiPZNWrDxspI9MBLYRnQYaw== 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.1] 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQC9XRdlqNNZGxAAUuQj7nDt7Mg2oqSETvgw7A== 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.2] 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDAXRdlBNr9HBAAPGa9EXA40S7bwfTHmlU4TQ== 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.3] 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDDXRdl8uIJHxAArpHRi4pHW12ga1CnmcSP7g== 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout:[client.0] 2023-09-29T23:29:10.856 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQC5XRdlLUE4DhAAwloMjGPQNpmB+ZmcCIQThA== 2023-09-29T23:29:10.857 INFO:teuthology.orchestra.run.smithi156.stdout:[client.1] 2023-09-29T23:29:10.857 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQC5XRdlflwmERAAa4kjNyBs6UVlfO4T8LhfTQ== 2023-09-29T23:29:10.857 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-29T23:29:10.860 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-29T23:29:10.900 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-29T23:29:10.946 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-29T23:29:10.996 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-29T23:29:10.998 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-29T23:29:11.045 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-29T23:29:11.047 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-29T23:29:11.091 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-29T23:29:11.131 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-29T23:29:11.177 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.z --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-29T23:29:11.179 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.z --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-29T23:29:11.268 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-29T23:29:11.270 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-29T23:29:11.318 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-29T23:29:11.320 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-29T23:29:11.363 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-29T23:29:11.365 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-29T23:29:11.449 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-29T23:29:11.451 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-29T23:29:11.493 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-29T23:29:11.495 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-29T23:29:11.537 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-29T23:29:11.582 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-29T23:29:11.630 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-09-29T23:29:11.630 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-09-29T23:29:11.647 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-29T23:29:11.701 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-09-29T23:29:11.714 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-09-29T23:29:11.728 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-29T23:29:11.832 DEBUG:teuthology.orchestra.run.smithi143:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-09-29T23:29:11.845 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-09-29T23:29:11.862 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-29T23:29:11.922 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-09-29T23:29:11.935 DEBUG:teuthology.orchestra.run.smithi143:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-29T23:29:11.937 DEBUG:teuthology.orchestra.run.smithi156:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-29T23:29:11.983 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-09-29T23:29:11.983 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-29T23:29:11.983 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-29T23:29:11.986 INFO:tasks.ceph.mon.a:Started 2023-09-29T23:29:11.986 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-29T23:29:11.986 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-29T23:29:11.988 INFO:tasks.ceph.mon.c:Started 2023-09-29T23:29:11.989 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-29T23:29:11.989 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-29T23:29:11.991 INFO:tasks.ceph.mon.b:Started 2023-09-29T23:29:11.991 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-09-29T23:29:11.991 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-29T23:29:11.992 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-29T23:29:11.994 INFO:tasks.ceph.mgr.x:Started 2023-09-29T23:29:11.994 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-09-29T23:29:11.994 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-09-29T23:29:11.997 INFO:tasks.ceph.mgr.y:Started 2023-09-29T23:29:11.997 INFO:tasks.ceph.mgr.z:Restarting daemon 2023-09-29T23:29:11.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i z 2023-09-29T23:29:11.999 INFO:tasks.ceph.mgr.z:Started 2023-09-29T23:29:11.999 INFO:tasks.ceph:Setting crush tunables to default 2023-09-29T23:29:12.000 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd crush tunables default 2023-09-29T23:29:12.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3f13e700 1 Processor -- start 2023-09-29T23:29:12.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3f13e700 1 -- start start 2023-09-29T23:29:12.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3f13e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:12.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3f13e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:12.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3f13e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3806f430 con 0x7f4a381411f0 2023-09-29T23:29:12.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3f13e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3809d3c0 con 0x7f4a3806f060 2023-09-29T23:29:12.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3f13e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3809d680 con 0x7f4a3809e980 2023-09-29T23:29:12.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:12.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:51258/0 (socket says 172.21.15.143:51258) 2023-09-29T23:29:12.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 learned_addr learned my addr 172.21.15.143:0/3561113173 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:12.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:12.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 msgr2=0x7f4a38140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-29T23:29:12.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 msgr2=0x7f4a38140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:12.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:12.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-29T23:29:12.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a3809e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-29T23:29:12.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a3809e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:12.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:12.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.093+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-29T23:29:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.297+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.297+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.297+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 msgr2=0x7f4a38140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-29T23:29:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.297+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 msgr2=0x7f4a38140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.297+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.297+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-29T23:29:12.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.297+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a3809e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-29T23:29:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.297+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a3809e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.297+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:12.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.297+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-29T23:29:12.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.701+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:12.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.701+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:12.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.701+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 msgr2=0x7f4a38140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-29T23:29:12.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.701+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 msgr2=0x7f4a38140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:12.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.701+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:12.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.701+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-29T23:29:12.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.701+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a3809e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-29T23:29:12.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.701+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a3809e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:12.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.701+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:12.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:12.701+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-29T23:29:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:13.505+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:13.505+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:13.505+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 msgr2=0x7f4a38140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-29T23:29:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:13.505+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 msgr2=0x7f4a38140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:13.505+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:13.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:13.505+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-29T23:29:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:13.505+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a3809e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-29T23:29:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:13.505+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a3809e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:13.505+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:13.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:13.505+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-29T23:29:15.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a367fc700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 msgr2=0x7f4a38140cb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-29T23:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a367fc700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a3809e980 0x7f4a38140cb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a367fc700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a3809e440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-29T23:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a367fc700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a3809e440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a367fc700 1 -- 172.21.15.143:0/3561113173 >> v1:172.21.15.156:6789/0 conn(0x7f4a381411f0 legacy=0x7f4a3809c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a367fc700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a367fc700 1 -- 172.21.15.143:0/3561113173 --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a3809dc60 con 0x7f4a24003710 2023-09-29T23:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a367fc700 1 -- 172.21.15.143:0/3561113173 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3809d3c0 con 0x7f4a24007400 2023-09-29T23:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-29T23:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:15.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.093+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-29T23:29:15.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.297+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:15.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.297+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-29T23:29:15.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.297+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:15.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.297+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:15.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.297+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-29T23:29:15.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.701+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.701+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-29T23:29:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.701+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.701+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:15.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:15.701+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-09-29T23:29:16.505 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:16.504+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:16.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:16.504+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-29T23:29:16.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:16.504+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:16.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:16.504+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:16.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:16.504+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-09-29T23:29:18.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a367fc700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-29T23:29:18.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a367fc700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a367fc700 1 -- 172.21.15.143:0/3561113173 >> v1:172.21.15.143:6789/0 conn(0x7f4a24003710 legacy=0x7f4a24005b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a367fc700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a3d6db700 1 -- 172.21.15.143:0/3561113173 reap_dead start 2023-09-29T23:29:18.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a367fc700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a2400a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a367fc700 1 -- 172.21.15.143:0/3561113173 --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2400b010 con 0x7f4a2400a5e0 2023-09-29T23:29:18.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a367fc700 1 -- 172.21.15.143:0/3561113173 --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3809d3c0 con 0x7f4a24007400 2023-09-29T23:29:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a367fc700 1 -- 172.21.15.143:0/3561113173 --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3809d680 con 0x7f4a3806f060 2023-09-29T23:29:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a2400a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-29T23:29:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-29T23:29:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a2400a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-29T23:29:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a2400a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a2400a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:18.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.092+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a2400a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-29T23:29:18.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.296+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.296+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-29T23:29:18.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.296+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:18.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.296+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a2400a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.296+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:18.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.296+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-29T23:29:18.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.296+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a2400a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-29T23:29:18.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.296+0000 7f4a37fff700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a2400a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-29T23:29:18.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.296+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a2400a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-29T23:29:18.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.296+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a2400a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-09-29T23:29:18.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.344+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836883570 0 0) 0x7f4a2400b010 con 0x7f4a2400a5e0 2023-09-29T23:29:18.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.344+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a20003570 con 0x7f4a2400a5e0 2023-09-29T23:29:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2418171792 0 0) 0x7f4a20003570 con 0x7f4a2400a5e0 2023-09-29T23:29:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 msgr2=0x7f4a2400a0a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-29T23:29:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.156:3300/0 conn(0x7f4a3806f060 0x7f4a2400a0a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 msgr2=0x7f4a240098b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-29T23:29:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 --2- 172.21.15.143:0/3561113173 >> v2:172.21.15.143:3300/0 conn(0x7f4a24007400 0x7f4a240098b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2400aab0 con 0x7f4a2400a5e0 2023-09-29T23:29:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251153714 0 0) 0x7f4a2c0013a0 con 0x7f4a2400a5e0 2023-09-29T23:29:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 >> v1:172.21.15.156:6789/0 conn(0x7f4a2400a5e0 legacy=0x7f4a24005b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a20003a60 0x7f4a20003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a20004810 0x7f4a20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a200051c0 0x7f4a2000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3809d3c0 con 0x7f4a20004810 2023-09-29T23:29:18.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3809d680 con 0x7f4a200051c0 2023-09-29T23:29:18.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2000c1b0 con 0x7f4a20003a60 2023-09-29T23:29:18.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a20003a60 0x7f4a20003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a3d6db700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a200051c0 0x7f4a2000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a20004810 0x7f4a20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a200051c0 msgr2=0x7f4a2000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:18.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a200051c0 0x7f4a2000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a20004810 msgr2=0x7f4a20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:18.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a20004810 0x7f4a20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a3ceda700 1 -- 172.21.15.143:0/3561113173 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a20003570 con 0x7f4a20003a60 2023-09-29T23:29:18.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a3d6db700 1 -- 172.21.15.143:0/3561113173 reap_dead start 2023-09-29T23:29:18.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a37fff700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a20004810 0x7f4a20004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:18.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.352+0000 7f4a3ceda700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a20003a60 0x7f4a20003e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800b310 tx=0x7f4a280106d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ed0fc7c199e7bd1 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:18.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.356+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a280109c0 con 0x7f4a20003a60 2023-09-29T23:29:18.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.356+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28017050 con 0x7f4a20003a60 2023-09-29T23:29:18.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.356+0000 7f4a377fe700 1 -- 172.21.15.143:0/3561113173 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a28011e60 con 0x7f4a20003a60 2023-09-29T23:29:18.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.356+0000 7f4a3f13e700 1 -- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a20003a60 msgr2=0x7f4a20003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.356+0000 7f4a3f13e700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a20003a60 0x7f4a20003e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800b310 tx=0x7f4a280106d0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.356+0000 7f4a3f13e700 1 -- 172.21.15.143:0/3561113173 shutdown_connections 2023-09-29T23:29:18.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.356+0000 7f4a3f13e700 1 --2- 172.21.15.143:0/3561113173 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a20003a60 0x7f4a20003e70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.356+0000 7f4a3f13e700 1 -- 172.21.15.143:0/3561113173 >> 172.21.15.143:0/3561113173 conn(0x7f4a3806a760 msgr2=0x7f4a380b2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:18.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.356+0000 7f4a3f13e700 1 -- 172.21.15.143:0/3561113173 shutdown_connections 2023-09-29T23:29:18.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.356+0000 7f4a3f13e700 1 -- 172.21.15.143:0/3561113173 wait complete. 2023-09-29T23:29:18.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a3f13e700 1 Processor -- start 2023-09-29T23:29:18.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a3f13e700 1 -- start start 2023-09-29T23:29:18.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a3f13e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a380a12d0 0x7f4a3813f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a3f13e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a380a16a0 0x7f4a3813dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a3f13e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a3813e1b0 0x7f4a3806b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a3f13e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3813ea00 con 0x7f4a3813e1b0 2023-09-29T23:29:18.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a3f13e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a38096ac0 con 0x7f4a380a12d0 2023-09-29T23:29:18.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a3f13e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a38096c70 con 0x7f4a380a16a0 2023-09-29T23:29:18.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a380a16a0 0x7f4a3813dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a37fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a380a16a0 0x7f4a3813dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:49948/0 (socket says 172.21.15.143:49948) 2023-09-29T23:29:18.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a3ceda700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a380a12d0 0x7f4a3813f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a37fff700 1 -- 172.21.15.143:0/4083631032 learned_addr learned my addr 172.21.15.143:0/4083631032 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:18.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a3d6db700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a3813e1b0 0x7f4a3806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a37fff700 1 -- 172.21.15.143:0/4083631032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a380a12d0 msgr2=0x7f4a3813f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a37fff700 1 --2- 172.21.15.143:0/4083631032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a380a12d0 0x7f4a3813f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a37fff700 1 -- 172.21.15.143:0/4083631032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a3813e1b0 msgr2=0x7f4a3806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a37fff700 1 --2- 172.21.15.143:0/4083631032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a3813e1b0 0x7f4a3806b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a37fff700 1 -- 172.21.15.143:0/4083631032 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3813f9b0 con 0x7f4a380a16a0 2023-09-29T23:29:18.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.360+0000 7f4a37fff700 1 --2- 172.21.15.143:0/4083631032 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a380a16a0 0x7f4a3813dc70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4a3001cdc0 tx=0x7f4a3001e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:18.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a34ff9700 1 -- 172.21.15.143:0/4083631032 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30021040 con 0x7f4a380a16a0 2023-09-29T23:29:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a34ff9700 1 -- 172.21.15.143:0/4083631032 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3000f040 con 0x7f4a380a16a0 2023-09-29T23:29:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a3f13e700 1 -- 172.21.15.143:0/4083631032 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3813fc70 con 0x7f4a380a16a0 2023-09-29T23:29:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a34ff9700 1 -- 172.21.15.143:0/4083631032 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30019e30 con 0x7f4a380a16a0 2023-09-29T23:29:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a3f13e700 1 -- 172.21.15.143:0/4083631032 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3813e650 con 0x7f4a380a16a0 2023-09-29T23:29:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a34ff9700 1 -- 172.21.15.143:0/4083631032 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f4a3001a650 con 0x7f4a380a16a0 2023-09-29T23:29:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a34ff9700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-09-29T23:29:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a34ff9700 4 mgrc handle_mgr_map Got map version 1 2023-09-29T23:29:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a34ff9700 4 mgrc handle_mgr_map Active mgr is now 2023-09-29T23:29:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a34ff9700 4 mgrc reconnect No active mgr available yet 2023-09-29T23:29:18.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a34ff9700 1 -- 172.21.15.143:0/4083631032 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4a3002f790 con 0x7f4a380a16a0 2023-09-29T23:29:18.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.364+0000 7f4a3f13e700 1 -- 172.21.15.143:0/4083631032 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3805ecb0 con 0x7f4a380a16a0 2023-09-29T23:29:18.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.368+0000 7f4a34ff9700 1 -- 172.21.15.143:0/4083631032 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f4a30034060 con 0x7f4a380a16a0 2023-09-29T23:29:18.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.436+0000 7f4a3f13e700 1 -- 172.21.15.143:0/4083631032 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4a38096db0 con 0x7f4a380a16a0 2023-09-29T23:29:18.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.444+0000 7f4a34ff9700 1 -- 172.21.15.143:0/4083631032 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4a3002a070 con 0x7f4a380a16a0 2023-09-29T23:29:18.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.448+0000 7f4a3f13e700 10 mgrc shutdown 2023-09-29T23:29:18.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.448+0000 7f4a3f13e700 1 -- 172.21.15.143:0/4083631032 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a380a16a0 msgr2=0x7f4a3813dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.448+0000 7f4a3f13e700 1 --2- 172.21.15.143:0/4083631032 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a380a16a0 0x7f4a3813dc70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4a3001cdc0 tx=0x7f4a3001e040 comp rx=0 tx=0).stop 2023-09-29T23:29:18.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.448+0000 7f4a3f13e700 1 -- 172.21.15.143:0/4083631032 shutdown_connections 2023-09-29T23:29:18.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.448+0000 7f4a3f13e700 1 --2- 172.21.15.143:0/4083631032 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4a3813e1b0 0x7f4a3806b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.448+0000 7f4a3f13e700 1 --2- 172.21.15.143:0/4083631032 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4a380a16a0 0x7f4a3813dc70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.448+0000 7f4a3f13e700 1 --2- 172.21.15.143:0/4083631032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a380a12d0 0x7f4a3813f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.448+0000 7f4a3f13e700 1 -- 172.21.15.143:0/4083631032 >> 172.21.15.143:0/4083631032 conn(0x7f4a3806a760 msgr2=0x7f4a3814bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:18.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.448+0000 7f4a3f13e700 1 -- 172.21.15.143:0/4083631032 shutdown_connections 2023-09-29T23:29:18.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:18.448+0000 7f4a3f13e700 1 -- 172.21.15.143:0/4083631032 wait complete. 2023-09-29T23:29:18.451 INFO:teuthology.orchestra.run.smithi143.stderr:adjusted tunables profile to default 2023-09-29T23:29:18.462 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-09-29T23:29:18.462 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2023-09-29T23:29:18.463 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-09-29T23:29:18.472 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:29:18.472 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-09-29T23:29:18.486 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:29:18.486 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-09-29T23:29:18.529 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:18.528+0000 7efc73251000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-29T23:29:18.544 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-09-29T23:29:18.544 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-09-29T23:29:18.551 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:18.549+0000 7fc7a7439000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-29T23:29:18.551 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:18.549+0000 7f4683d0a000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-29T23:29:18.553 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new c268b844-dbdc-482b-965f-f38c2a1da791 0 2023-09-29T23:29:18.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.649+0000 7f0370d10700 1 Processor -- start 2023-09-29T23:29:18.653 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.649+0000 7f0370d10700 1 -- start start 2023-09-29T23:29:18.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f036c09d110 0x7f036c09dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f036c0a5d00 0x7f036c147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c0a4ac0 con 0x7f036c06f060 2023-09-29T23:29:18.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c09ea00 con 0x7f036c0a5d00 2023-09-29T23:29:18.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c06f430 con 0x7f036c09d110 2023-09-29T23:29:18.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f036bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f036c0a5d00 0x7f036c147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f036bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f036c0a5d00 0x7f036c147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39176/0 (socket says 172.21.15.156:39176) 2023-09-29T23:29:18.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f036bfff700 1 -- 172.21.15.156:0/1645550798 learned_addr learned my addr 172.21.15.156:0/1645550798 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-29T23:29:18.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0363fff700 1 --2- 172.21.15.156:0/1645550798 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f036c09d110 0x7f036c09dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f036bfff700 1 -- 172.21.15.156:0/1645550798 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f036c09d110 msgr2=0x7f036c09dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:18.655 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f036bfff700 1 --2- 172.21.15.156:0/1645550798 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f036c09d110 0x7f036c09dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f036bfff700 1 -- 172.21.15.156:0/1645550798 >> v1:172.21.15.143:6789/0 conn(0x7f036c06f060 legacy=0x7f036c140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f036bfff700 1 -- 172.21.15.156:0/1645550798 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c09e510 con 0x7f036c0a5d00 2023-09-29T23:29:18.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f036bfff700 1 --2- 172.21.15.156:0/1645550798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f036c0a5d00 0x7f036c147730 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0364009f20 tx=0x7f0364009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f024191c634ca033 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:18.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f036affd700 1 -- 172.21.15.156:0/1645550798 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036400e070 con 0x7f036c0a5d00 2023-09-29T23:29:18.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f036affd700 1 -- 172.21.15.156:0/1645550798 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036400ba80 con 0x7f036c0a5d00 2023-09-29T23:29:18.656 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f036affd700 1 -- 172.21.15.156:0/1645550798 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03640092e0 con 0x7f036c0a5d00 2023-09-29T23:29:18.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 -- 172.21.15.156:0/1645550798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f036c0a5d00 msgr2=0x7f036c147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 --2- 172.21.15.156:0/1645550798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f036c0a5d00 0x7f036c147730 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0364009f20 tx=0x7f0364009840 comp rx=0 tx=0).stop 2023-09-29T23:29:18.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 -- 172.21.15.156:0/1645550798 shutdown_connections 2023-09-29T23:29:18.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 --2- 172.21.15.156:0/1645550798 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f036c09d110 0x7f036c09dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 --2- 172.21.15.156:0/1645550798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f036c0a5d00 0x7f036c147730 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.657 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 -- 172.21.15.156:0/1645550798 >> 172.21.15.156:0/1645550798 conn(0x7f036c06a730 msgr2=0x7f036c0b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:18.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 -- 172.21.15.156:0/1645550798 shutdown_connections 2023-09-29T23:29:18.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 -- 172.21.15.156:0/1645550798 wait complete. 2023-09-29T23:29:18.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 Processor -- start 2023-09-29T23:29:18.658 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.653+0000 7f0370d10700 1 -- start start 2023-09-29T23:29:18.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0370d10700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f036c06f060 0x7f036c06cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.660 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0370d10700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f036c09d110 0x7f036c14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0370d10700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036c0a5d00 0x7f036c1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0370d10700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c150db0 con 0x7f036c0a5d00 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0370d10700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c1508e0 con 0x7f036c09d110 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0370d10700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0a2a00 con 0x7f036c06f060 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0363fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f036c09d110 0x7f036c14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0363fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f036c09d110 0x7f036c14fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39178/0 (socket says 172.21.15.156:39178) 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0363fff700 1 -- 172.21.15.156:0/3434740096 learned_addr learned my addr 172.21.15.156:0/3434740096 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f036bfff700 1 --2- 172.21.15.156:0/3434740096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036c0a5d00 0x7f036c1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f036b7fe700 1 --2- 172.21.15.156:0/3434740096 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f036c06f060 0x7f036c06cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0363fff700 1 -- 172.21.15.156:0/3434740096 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f036c06f060 msgr2=0x7f036c06cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0363fff700 1 --2- 172.21.15.156:0/3434740096 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f036c06f060 0x7f036c06cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.661 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0363fff700 1 -- 172.21.15.156:0/3434740096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036c0a5d00 msgr2=0x7f036c1503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0363fff700 1 --2- 172.21.15.156:0/3434740096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036c0a5d00 0x7f036c1503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0363fff700 1 -- 172.21.15.156:0/3434740096 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0364009ae0 con 0x7f036c09d110 2023-09-29T23:29:18.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0363fff700 1 --2- 172.21.15.156:0/3434740096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f036c09d110 0x7f036c14fe60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f035400d8c0 tx=0x7f0354008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:18.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f03637fe700 1 -- 172.21.15.156:0/3434740096 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035400de60 con 0x7f036c09d110 2023-09-29T23:29:18.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f03637fe700 1 -- 172.21.15.156:0/3434740096 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035400b780 con 0x7f036c09d110 2023-09-29T23:29:18.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0370d10700 1 -- 172.21.15.156:0/3434740096 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036c06c380 con 0x7f036c09d110 2023-09-29T23:29:18.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f03637fe700 1 -- 172.21.15.156:0/3434740096 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035400bb20 con 0x7f036c09d110 2023-09-29T23:29:18.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0370d10700 1 -- 172.21.15.156:0/3434740096 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c09fcb0 con 0x7f036c09d110 2023-09-29T23:29:18.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f03637fe700 1 -- 172.21.15.156:0/3434740096 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f035401e460 con 0x7f036c09d110 2023-09-29T23:29:18.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f03637fe700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-09-29T23:29:18.662 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f03637fe700 4 mgrc handle_mgr_map Got map version 1 2023-09-29T23:29:18.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f03637fe700 4 mgrc handle_mgr_map Active mgr is now 2023-09-29T23:29:18.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f03637fe700 4 mgrc reconnect No active mgr available yet 2023-09-29T23:29:18.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f03637fe700 1 -- 172.21.15.156:0/3434740096 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0354014780 con 0x7f036c09d110 2023-09-29T23:29:18.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.657+0000 7f0370d10700 1 -- 172.21.15.156:0/3434740096 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f036c05ecb0 con 0x7f036c09d110 2023-09-29T23:29:18.663 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.661+0000 7f03637fe700 1 -- 172.21.15.156:0/3434740096 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f035401e8a0 con 0x7f036c09d110 2023-09-29T23:29:18.740 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.737+0000 7f0370d10700 1 -- 172.21.15.156:0/3434740096 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c268b844-dbdc-482b-965f-f38c2a1da791", "id": 0} v 0) v1 -- 0x7f036c09d7f0 con 0x7f036c09d110 2023-09-29T23:29:18.745 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.745+0000 7f03637fe700 1 -- 172.21.15.156:0/3434740096 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c268b844-dbdc-482b-965f-f38c2a1da791", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f035402d0b0 con 0x7f036c09d110 2023-09-29T23:29:18.746 INFO:teuthology.orchestra.run.smithi156.stdout:0 2023-09-29T23:29:18.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.745+0000 7f0370d10700 10 mgrc shutdown 2023-09-29T23:29:18.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.745+0000 7f0370d10700 1 -- 172.21.15.156:0/3434740096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f036c09d110 msgr2=0x7f036c14fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.745+0000 7f0370d10700 1 --2- 172.21.15.156:0/3434740096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f036c09d110 0x7f036c14fe60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f035400d8c0 tx=0x7f0354008040 comp rx=0 tx=0).stop 2023-09-29T23:29:18.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.745+0000 7f0370d10700 1 -- 172.21.15.156:0/3434740096 shutdown_connections 2023-09-29T23:29:18.747 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.745+0000 7f0370d10700 1 --2- 172.21.15.156:0/3434740096 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f036c0a5d00 0x7f036c1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.745+0000 7f0370d10700 1 --2- 172.21.15.156:0/3434740096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f036c09d110 0x7f036c14fe60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.745+0000 7f0370d10700 1 --2- 172.21.15.156:0/3434740096 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f036c06f060 0x7f036c06cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.745+0000 7f0370d10700 1 -- 172.21.15.156:0/3434740096 >> 172.21.15.156:0/3434740096 conn(0x7f036c06a730 msgr2=0x7f036c06d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:18.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.745+0000 7f0370d10700 1 -- 172.21.15.156:0/3434740096 shutdown_connections 2023-09-29T23:29:18.748 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.745+0000 7f0370d10700 1 -- 172.21.15.156:0/3434740096 wait complete. 2023-09-29T23:29:18.759 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new ab4ecead-b2d8-4c5f-849e-fc69aee684ed 1 2023-09-29T23:29:18.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.857+0000 7fd6011ce700 1 Processor -- start 2023-09-29T23:29:18.860 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.857+0000 7fd6011ce700 1 -- start start 2023-09-29T23:29:18.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.857+0000 7fd6011ce700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5fc0a8900 0x7fd5fc067120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.857+0000 7fd6011ce700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc141420 con 0x7fd5fc0a9970 2023-09-29T23:29:18.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.857+0000 7fd6011ce700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5fc1416e0 con 0x7fd5fc141cc0 2023-09-29T23:29:18.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.857+0000 7fd6011ce700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0aa340 con 0x7fd5fc0a8900 2023-09-29T23:29:18.861 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.857+0000 7fd5fad9d700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fd5fc141cc0 0x7fd5fc0a7510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37044/0 (socket says 172.21.15.156:37044) 2023-09-29T23:29:18.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.857+0000 7fd5fad9d700 1 -- 172.21.15.156:0/1549080732 learned_addr learned my addr 172.21.15.156:0/1549080732 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-09-29T23:29:18.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.857+0000 7fd5fa59c700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5fc0a8900 0x7fd5fc067120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.857+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407725675 0 0) 0x7fd5fc1416e0 con 0x7fd5fc141cc0 2023-09-29T23:29:18.862 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.857+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5e4003570 con 0x7fd5fc141cc0 2023-09-29T23:29:18.864 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366363463 0 0) 0x7fd5fc141420 con 0x7fd5fc0a9970 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5fc1416e0 con 0x7fd5fc0a9970 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4230916859 0 0) 0x7fd5e4003570 con 0x7fd5fc141cc0 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5fc0a8900 msgr2=0x7fd5fc067120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5fc0a8900 0x7fd5fc067120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 >> v1:172.21.15.143:6789/0 conn(0x7fd5fc0a9970 legacy=0x7fd5fc148640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc067660 con 0x7fd5fc141cc0 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251153714 0 0) 0x7fd5ec001470 con 0x7fd5fc141cc0 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 >> v1:172.21.15.156:6789/0 conn(0x7fd5fc141cc0 legacy=0x7fd5fc0a7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5e4003c20 0x7fd5e4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5fc0a8900 0x7fd5e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5e4004d50 0x7fd5e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.865 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc0aa340 con 0x7fd5e4004d50 2023-09-29T23:29:18.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e400bb70 con 0x7fd5e4003c20 2023-09-29T23:29:18.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e400bd30 con 0x7fd5fc0a8900 2023-09-29T23:29:18.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5fad9d700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5e4003c20 0x7fd5e4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5fa59c700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5fc0a8900 0x7fd5e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5fad9d700 1 -- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5fc0a8900 msgr2=0x7fd5e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:18.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5fb59e700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5e4004d50 0x7fd5e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5fad9d700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5fc0a8900 0x7fd5e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5fad9d700 1 -- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5e4004d50 msgr2=0x7fd5e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:18.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5fad9d700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5e4004d50 0x7fd5e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5fad9d700 1 -- 172.21.15.156:0/1549080732 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e4003570 con 0x7fd5e4003c20 2023-09-29T23:29:18.866 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5fb59e700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5e4004d50 0x7fd5e40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-29T23:29:18.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5fad9d700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5e4003c20 0x7fd5e4004010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00c2d0 tx=0x7fd5ec001e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18efcd76eb75a4a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:18.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec00cc00 con 0x7fd5e4003c20 2023-09-29T23:29:18.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ec00cd60 con 0x7fd5e4003c20 2023-09-29T23:29:18.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5f9d9b700 1 -- 172.21.15.156:0/1549080732 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec005b70 con 0x7fd5e4003c20 2023-09-29T23:29:18.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd6011ce700 1 -- 172.21.15.156:0/1549080732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5e4003c20 msgr2=0x7fd5e4004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd6011ce700 1 --2- 172.21.15.156:0/1549080732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5e4003c20 0x7fd5e4004010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00c2d0 tx=0x7fd5ec001e10 comp rx=0 tx=0).stop 2023-09-29T23:29:18.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd5fb59e700 1 -- 172.21.15.156:0/1549080732 reap_dead start 2023-09-29T23:29:18.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd6011ce700 1 -- 172.21.15.156:0/1549080732 shutdown_connections 2023-09-29T23:29:18.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd6011ce700 1 -- 172.21.15.156:0/1549080732 >> 172.21.15.156:0/1549080732 conn(0x7fd5fc06bda0 msgr2=0x7fd5fc06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:18.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd6011ce700 1 -- 172.21.15.156:0/1549080732 shutdown_connections 2023-09-29T23:29:18.867 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.861+0000 7fd6011ce700 1 -- 172.21.15.156:0/1549080732 wait complete. 2023-09-29T23:29:18.869 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd6011ce700 1 Processor -- start 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd6011ce700 1 -- start start 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd6011ce700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5e4003c20 0x7fd5fc06b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd6011ce700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5fc06b580 0x7fd5fc11d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd6011ce700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5fc11d960 0x7fd5fc139e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd6011ce700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc11cd20 con 0x7fd5fc06b580 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd6011ce700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc13aab0 con 0x7fd5fc11d960 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd6011ce700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5fc14ff00 con 0x7fd5e4003c20 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5fb59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5fc11d960 0x7fd5fc139e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5fad9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5e4003c20 0x7fd5fc06b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5fb59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5fc11d960 0x7fd5fc139e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39204/0 (socket says 172.21.15.156:39204) 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5fb59e700 1 -- 172.21.15.156:0/638488496 learned_addr learned my addr 172.21.15.156:0/638488496 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-29T23:29:18.870 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5fa59c700 1 --2- 172.21.15.156:0/638488496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5fc06b580 0x7fd5fc11d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5fb59e700 1 -- 172.21.15.156:0/638488496 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5e4003c20 msgr2=0x7fd5fc06b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5fb59e700 1 --2- 172.21.15.156:0/638488496 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5e4003c20 0x7fd5fc06b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5fb59e700 1 -- 172.21.15.156:0/638488496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5fc06b580 msgr2=0x7fd5fc11d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5fb59e700 1 --2- 172.21.15.156:0/638488496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5fc06b580 0x7fd5fc11d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5fb59e700 1 -- 172.21.15.156:0/638488496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc11ddd0 con 0x7fd5fc11d960 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5fb59e700 1 --2- 172.21.15.156:0/638488496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5fc11d960 0x7fd5fc139e20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc06c920 tx=0x7fd5f4009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5e2ffd700 1 -- 172.21.15.156:0/638488496 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4003300 con 0x7fd5fc11d960 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5e2ffd700 1 -- 172.21.15.156:0/638488496 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f4003460 con 0x7fd5fc11d960 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd6011ce700 1 -- 172.21.15.156:0/638488496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5fc150170 con 0x7fd5fc11d960 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5e2ffd700 1 -- 172.21.15.156:0/638488496 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4006df0 con 0x7fd5fc11d960 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5e2ffd700 1 -- 172.21.15.156:0/638488496 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fd5f4001b70 con 0x7fd5fc11d960 2023-09-29T23:29:18.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5e2ffd700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-09-29T23:29:18.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5e2ffd700 4 mgrc handle_mgr_map Got map version 1 2023-09-29T23:29:18.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5e2ffd700 4 mgrc handle_mgr_map Active mgr is now 2023-09-29T23:29:18.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd5e2ffd700 4 mgrc reconnect No active mgr available yet 2023-09-29T23:29:18.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.865+0000 7fd6011ce700 1 -- 172.21.15.156:0/638488496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5fc11c6d0 con 0x7fd5fc11d960 2023-09-29T23:29:18.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.869+0000 7fd5e2ffd700 1 -- 172.21.15.156:0/638488496 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fd5f40078a0 con 0x7fd5fc11d960 2023-09-29T23:29:18.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.869+0000 7fd6011ce700 1 -- 172.21.15.156:0/638488496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5fc06e0a0 con 0x7fd5fc11d960 2023-09-29T23:29:18.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.873+0000 7fd5e2ffd700 1 -- 172.21.15.156:0/638488496 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fd5fc06e0a0 con 0x7fd5fc11d960 2023-09-29T23:29:18.949 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.945+0000 7fd6011ce700 1 -- 172.21.15.156:0/638488496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ab4ecead-b2d8-4c5f-849e-fc69aee684ed", "id": 1} v 0) v1 -- 0x7fd5fc139b40 con 0x7fd5fc11d960 2023-09-29T23:29:18.956 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.953+0000 7fd5e2ffd700 1 -- 172.21.15.156:0/638488496 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ab4ecead-b2d8-4c5f-849e-fc69aee684ed", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd5fc139b40 con 0x7fd5fc11d960 2023-09-29T23:29:18.956 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-09-29T23:29:18.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.953+0000 7fd6011ce700 10 mgrc shutdown 2023-09-29T23:29:18.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.953+0000 7fd6011ce700 1 -- 172.21.15.156:0/638488496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5fc11d960 msgr2=0x7fd5fc139e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:18.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.953+0000 7fd6011ce700 1 --2- 172.21.15.156:0/638488496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5fc11d960 0x7fd5fc139e20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc06c920 tx=0x7fd5f4009040 comp rx=0 tx=0).stop 2023-09-29T23:29:18.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.957+0000 7fd6011ce700 1 -- 172.21.15.156:0/638488496 shutdown_connections 2023-09-29T23:29:18.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.957+0000 7fd6011ce700 1 --2- 172.21.15.156:0/638488496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd5fc11d960 0x7fd5fc139e20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.957+0000 7fd6011ce700 1 --2- 172.21.15.156:0/638488496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd5fc06b580 0x7fd5fc11d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.957+0000 7fd6011ce700 1 --2- 172.21.15.156:0/638488496 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd5e4003c20 0x7fd5fc06b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:18.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.957+0000 7fd6011ce700 1 -- 172.21.15.156:0/638488496 >> 172.21.15.156:0/638488496 conn(0x7fd5fc06bda0 msgr2=0x7fd5fc13ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:18.958 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.957+0000 7fd6011ce700 1 -- 172.21.15.156:0/638488496 shutdown_connections 2023-09-29T23:29:18.959 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:18.957+0000 7fd6011ce700 1 -- 172.21.15.156:0/638488496 wait complete. 2023-09-29T23:29:18.974 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 907c543a-0ea3-4163-84a6-f50f5ed849fb 2 2023-09-29T23:29:19.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.065+0000 7f12add3e700 1 Processor -- start 2023-09-29T23:29:19.069 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.065+0000 7f12add3e700 1 -- start start 2023-09-29T23:29:19.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12a806ee30 0x7f12a80b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12a80a5b80 0x7f12a80a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a80a4af0 con 0x7f12a806f200 2023-09-29T23:29:19.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a80a2e50 con 0x7f12a80a5b80 2023-09-29T23:29:19.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a80a6020 con 0x7f12a806ee30 2023-09-29T23:29:19.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12ad53d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12a80a5b80 0x7f12a80a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.071 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12ad53d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12a80a5b80 0x7f12a80a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39216/0 (socket says 172.21.15.156:39216) 2023-09-29T23:29:19.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12ad53d700 1 -- 172.21.15.156:0/858375156 learned_addr learned my addr 172.21.15.156:0/858375156 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-29T23:29:19.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12acd3c700 1 --2- 172.21.15.156:0/858375156 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12a806ee30 0x7f12a80b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12ad53d700 1 -- 172.21.15.156:0/858375156 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12a806ee30 msgr2=0x7f12a80b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:19.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12ad53d700 1 --2- 172.21.15.156:0/858375156 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12a806ee30 0x7f12a80b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12ad53d700 1 -- 172.21.15.156:0/858375156 >> v1:172.21.15.143:6789/0 conn(0x7f12a806f200 legacy=0x7f12a813d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:19.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12ad53d700 1 -- 172.21.15.156:0/858375156 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a809c6f0 con 0x7f12a80a5b80 2023-09-29T23:29:19.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12ad53d700 1 --2- 172.21.15.156:0/858375156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a80a5b80 0x7f12a80a45b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f12980099b0 tx=0x7f1298009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=648db4cb947c14b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:19.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12a77fe700 1 -- 172.21.15.156:0/858375156 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129800b8f0 con 0x7f12a80a5b80 2023-09-29T23:29:19.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12a77fe700 1 -- 172.21.15.156:0/858375156 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129800ba50 con 0x7f12a80a5b80 2023-09-29T23:29:19.072 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12a77fe700 1 -- 172.21.15.156:0/858375156 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129800bdf0 con 0x7f12a80a5b80 2023-09-29T23:29:19.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 -- 172.21.15.156:0/858375156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a80a5b80 msgr2=0x7f12a80a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.075 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 --2- 172.21.15.156:0/858375156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a80a5b80 0x7f12a80a45b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f12980099b0 tx=0x7f1298009250 comp rx=0 tx=0).stop 2023-09-29T23:29:19.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 -- 172.21.15.156:0/858375156 shutdown_connections 2023-09-29T23:29:19.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 --2- 172.21.15.156:0/858375156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a80a5b80 0x7f12a80a45b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 --2- 172.21.15.156:0/858375156 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12a806ee30 0x7f12a80b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 -- 172.21.15.156:0/858375156 >> 172.21.15.156:0/858375156 conn(0x7f12a806a730 msgr2=0x7f12a80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:19.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 -- 172.21.15.156:0/858375156 shutdown_connections 2023-09-29T23:29:19.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 -- 172.21.15.156:0/858375156 wait complete. 2023-09-29T23:29:19.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 Processor -- start 2023-09-29T23:29:19.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.069+0000 7f12add3e700 1 -- start start 2023-09-29T23:29:19.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12add3e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a806ee30 0x7f12a8143750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12add3e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12a806f200 0x7f12a8152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.076 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12add3e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a80a5b80 0x7f12a809d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12add3e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a80969d0 con 0x7f12a806ee30 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12add3e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a8153740 con 0x7f12a80a5b80 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12add3e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a8153a00 con 0x7f12a806f200 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12ad53d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a80a5b80 0x7f12a809d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12ad53d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a80a5b80 0x7f12a809d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:39222/0 (socket says 172.21.15.156:39222) 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12ad53d700 1 -- 172.21.15.156:0/3774285597 learned_addr learned my addr 172.21.15.156:0/3774285597 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12a7fff700 1 --2- 172.21.15.156:0/3774285597 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12a806f200 0x7f12a8152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12acd3c700 1 --2- 172.21.15.156:0/3774285597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a806ee30 0x7f12a8143750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12ad53d700 1 -- 172.21.15.156:0/3774285597 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12a806f200 msgr2=0x7f12a8152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12ad53d700 1 --2- 172.21.15.156:0/3774285597 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12a806f200 0x7f12a8152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12ad53d700 1 -- 172.21.15.156:0/3774285597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a806ee30 msgr2=0x7f12a8143750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.077 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12ad53d700 1 --2- 172.21.15.156:0/3774285597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a806ee30 0x7f12a8143750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12ad53d700 1 -- 172.21.15.156:0/3774285597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12980094f0 con 0x7f12a80a5b80 2023-09-29T23:29:19.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12a7fff700 1 --2- 172.21.15.156:0/3774285597 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12a806f200 0x7f12a8152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:19.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12ad53d700 1 --2- 172.21.15.156:0/3774285597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a80a5b80 0x7f12a809d920 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f12a80a4c80 tx=0x7f1298017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:19.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12a4ff9700 1 -- 172.21.15.156:0/3774285597 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129801a660 con 0x7f12a80a5b80 2023-09-29T23:29:19.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12a4ff9700 1 -- 172.21.15.156:0/3774285597 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129801a7c0 con 0x7f12a80a5b80 2023-09-29T23:29:19.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12add3e700 1 -- 172.21.15.156:0/3774285597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a8095ae0 con 0x7f12a80a5b80 2023-09-29T23:29:19.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12a4ff9700 1 -- 172.21.15.156:0/3774285597 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129801ab80 con 0x7f12a80a5b80 2023-09-29T23:29:19.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12add3e700 1 -- 172.21.15.156:0/3774285597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a813a8c0 con 0x7f12a80a5b80 2023-09-29T23:29:19.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12a4ff9700 1 -- 172.21.15.156:0/3774285597 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f129802e6c0 con 0x7f12a80a5b80 2023-09-29T23:29:19.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12a4ff9700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-09-29T23:29:19.078 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12a4ff9700 4 mgrc handle_mgr_map Got map version 1 2023-09-29T23:29:19.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12a4ff9700 4 mgrc handle_mgr_map Active mgr is now 2023-09-29T23:29:19.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12a4ff9700 4 mgrc reconnect No active mgr available yet 2023-09-29T23:29:19.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12a4ff9700 1 -- 172.21.15.156:0/3774285597 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f1298025870 con 0x7f12a80a5b80 2023-09-29T23:29:19.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.073+0000 7f12add3e700 1 -- 172.21.15.156:0/3774285597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a80966d0 con 0x7f12a80a5b80 2023-09-29T23:29:19.079 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.077+0000 7f12a4ff9700 1 -- 172.21.15.156:0/3774285597 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f1298015020 con 0x7f12a80a5b80 2023-09-29T23:29:19.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.149+0000 7f12add3e700 1 -- 172.21.15.156:0/3774285597 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "907c543a-0ea3-4163-84a6-f50f5ed849fb", "id": 2} v 0) v1 -- 0x7f12a805ecb0 con 0x7f12a80a5b80 2023-09-29T23:29:19.158 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.157+0000 7f12a4ff9700 1 -- 172.21.15.156:0/3774285597 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "907c543a-0ea3-4163-84a6-f50f5ed849fb", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1298025440 con 0x7f12a80a5b80 2023-09-29T23:29:19.158 INFO:teuthology.orchestra.run.smithi156.stdout:2 2023-09-29T23:29:19.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.157+0000 7f12add3e700 10 mgrc shutdown 2023-09-29T23:29:19.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.157+0000 7f12add3e700 1 -- 172.21.15.156:0/3774285597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a80a5b80 msgr2=0x7f12a809d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.157+0000 7f12add3e700 1 --2- 172.21.15.156:0/3774285597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a80a5b80 0x7f12a809d920 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f12a80a4c80 tx=0x7f1298017000 comp rx=0 tx=0).stop 2023-09-29T23:29:19.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.157+0000 7f12add3e700 1 -- 172.21.15.156:0/3774285597 shutdown_connections 2023-09-29T23:29:19.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.157+0000 7f12add3e700 1 --2- 172.21.15.156:0/3774285597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12a80a5b80 0x7f12a809d920 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.157+0000 7f12add3e700 1 --2- 172.21.15.156:0/3774285597 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12a806f200 0x7f12a8152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.159 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.157+0000 7f12add3e700 1 --2- 172.21.15.156:0/3774285597 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12a806ee30 0x7f12a8143750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.157+0000 7f12add3e700 1 -- 172.21.15.156:0/3774285597 >> 172.21.15.156:0/3774285597 conn(0x7f12a806a730 msgr2=0x7f12a813e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:19.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.157+0000 7f12add3e700 1 -- 172.21.15.156:0/3774285597 shutdown_connections 2023-09-29T23:29:19.160 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.157+0000 7f12add3e700 1 -- 172.21.15.156:0/3774285597 wait complete. 2023-09-29T23:29:19.171 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new bf5a122c-0873-4890-8845-702c4fc3325b 3 2023-09-29T23:29:19.173 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:19.172+0000 7efc73251000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-29T23:29:19.204 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:19.201+0000 7f4683d0a000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-29T23:29:19.205 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:19.201+0000 7fc7a7439000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-29T23:29:19.230 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:19.228+0000 7efc73251000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-29T23:29:19.262 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:19.261+0000 7f4683d0a000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-29T23:29:19.262 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:19.261+0000 7fc7a7439000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-29T23:29:19.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.269+0000 7f584359e700 1 Processor -- start 2023-09-29T23:29:19.272 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.269+0000 7f584359e700 1 -- start start 2023-09-29T23:29:19.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.269+0000 7f584359e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f58440b0560 0x7f58440a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.273 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.269+0000 7f584359e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58440b3d60 con 0x7f58440b3990 2023-09-29T23:29:19.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.269+0000 7f584359e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5844140850 con 0x7f58440b0d70 2023-09-29T23:29:19.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.269+0000 7f584359e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58440a6360 con 0x7f58440b0560 2023-09-29T23:29:19.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.269+0000 7f583bfff700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f58440b0d70 0x7f584413ff20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:37046/0 (socket says 172.21.15.156:37046) 2023-09-29T23:29:19.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.269+0000 7f583bfff700 1 -- 172.21.15.156:0/3254200532 learned_addr learned my addr 172.21.15.156:0/3254200532 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-09-29T23:29:19.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5842d9d700 1 --2- 172.21.15.156:0/3254200532 >> v2:172.21.15.143:3300/0 conn(0x7f58440b0560 0x7f58440a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785674224 0 0) 0x7f5844140850 con 0x7f58440b0d70 2023-09-29T23:29:19.274 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5830003570 con 0x7f58440b0d70 2023-09-29T23:29:19.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 748286233 0 0) 0x7f5830003570 con 0x7f58440b0d70 2023-09-29T23:29:19.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 >> v2:172.21.15.143:3300/0 conn(0x7f58440b0560 msgr2=0x7f58440a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:19.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 --2- 172.21.15.156:0/3254200532 >> v2:172.21.15.143:3300/0 conn(0x7f58440b0560 0x7f58440a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 >> v1:172.21.15.143:6789/0 conn(0x7f58440b3990 legacy=0x7f58440a5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5844140b10 con 0x7f58440b0d70 2023-09-29T23:29:19.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251153714 0 0) 0x7f582c001470 con 0x7f58440b0d70 2023-09-29T23:29:19.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 >> v1:172.21.15.156:6789/0 conn(0x7f58440b0d70 legacy=0x7f584413ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 --2- 172.21.15.156:0/3254200532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830003a60 0x7f5830003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.277 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 --2- 172.21.15.156:0/3254200532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5830004810 0x7f5830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 --2- 172.21.15.156:0/3254200532 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f58300051c0 0x7f583000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58440a6360 con 0x7f5830003a60 2023-09-29T23:29:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583000c0d0 con 0x7f5830004810 2023-09-29T23:29:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583000c290 con 0x7f58300051c0 2023-09-29T23:29:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5842d9d700 1 --2- 172.21.15.156:0/3254200532 >> v2:172.21.15.143:3300/0 conn(0x7f58440b0560 0x7f58440a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f583bfff700 1 --2- 172.21.15.156:0/3254200532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5830004810 0x7f5830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f584259c700 1 --2- 172.21.15.156:0/3254200532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830003a60 0x7f5830003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5842d9d700 1 --2- 172.21.15.156:0/3254200532 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f58300051c0 0x7f583000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f583bfff700 1 -- 172.21.15.156:0/3254200532 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f58300051c0 msgr2=0x7f583000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f583bfff700 1 --2- 172.21.15.156:0/3254200532 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f58300051c0 0x7f583000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.278 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f583bfff700 1 -- 172.21.15.156:0/3254200532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830003a60 msgr2=0x7f5830003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f583bfff700 1 --2- 172.21.15.156:0/3254200532 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830003a60 0x7f5830003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f583bfff700 1 -- 172.21.15.156:0/3254200532 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5830003570 con 0x7f5830004810 2023-09-29T23:29:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5842d9d700 1 -- 172.21.15.156:0/3254200532 reap_dead start 2023-09-29T23:29:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f583bfff700 1 --2- 172.21.15.156:0/3254200532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5830004810 0x7f5830004c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f582c00c3c0 tx=0x7f582c001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a15ba6778aaff7bf server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c00cc90 con 0x7f5830004810 2023-09-29T23:29:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c00ce10 con 0x7f5830004810 2023-09-29T23:29:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f5841d9b700 1 -- 172.21.15.156:0/3254200532 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c005b70 con 0x7f5830004810 2023-09-29T23:29:19.279 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f584359e700 1 -- 172.21.15.156:0/3254200532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5830004810 msgr2=0x7f5830004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f584359e700 1 --2- 172.21.15.156:0/3254200532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5830004810 0x7f5830004c80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f582c00c3c0 tx=0x7f582c001e50 comp rx=0 tx=0).stop 2023-09-29T23:29:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f584359e700 1 -- 172.21.15.156:0/3254200532 shutdown_connections 2023-09-29T23:29:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f584359e700 1 --2- 172.21.15.156:0/3254200532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5830004810 0x7f5830004c80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f584359e700 1 -- 172.21.15.156:0/3254200532 >> 172.21.15.156:0/3254200532 conn(0x7f584406ab00 msgr2=0x7f584406aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f584359e700 1 -- 172.21.15.156:0/3254200532 shutdown_connections 2023-09-29T23:29:19.280 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.273+0000 7f584359e700 1 -- 172.21.15.156:0/3254200532 wait complete. 2023-09-29T23:29:19.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f584359e700 1 Processor -- start 2023-09-29T23:29:19.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f584359e700 1 -- start start 2023-09-29T23:29:19.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f584359e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830004810 0x7f58440aeda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f584359e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f584413e340 0x7f584413e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f584359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58441504c0 0x7f5844150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f584359e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5844144250 con 0x7f5830004810 2023-09-29T23:29:19.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f584359e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584413ecd0 con 0x7f58441504c0 2023-09-29T23:29:19.283 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f584359e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5844150200 con 0x7f584413e340 2023-09-29T23:29:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f583bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f584413e340 0x7f584413e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f5842d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58441504c0 0x7f5844150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f583bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f584413e340 0x7f584413e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.156:49510/0 (socket says 172.21.15.156:49510) 2023-09-29T23:29:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f583bfff700 1 -- 172.21.15.156:0/3058587857 learned_addr learned my addr 172.21.15.156:0/3058587857 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-09-29T23:29:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f584259c700 1 --2- 172.21.15.156:0/3058587857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830004810 0x7f58440aeda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f583bfff700 1 -- 172.21.15.156:0/3058587857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58441504c0 msgr2=0x7f5844150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f583bfff700 1 --2- 172.21.15.156:0/3058587857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58441504c0 0x7f5844150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f583bfff700 1 -- 172.21.15.156:0/3058587857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830004810 msgr2=0x7f58440aeda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f583bfff700 1 --2- 172.21.15.156:0/3058587857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830004810 0x7f58440aeda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f583bfff700 1 -- 172.21.15.156:0/3058587857 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440adcb0 con 0x7f584413e340 2023-09-29T23:29:19.284 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f5842d9d700 1 --2- 172.21.15.156:0/3058587857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58441504c0 0x7f5844150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.277+0000 7f583bfff700 1 --2- 172.21.15.156:0/3058587857 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f584413e340 0x7f584413e790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f583000c260 tx=0x7f582c015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f583a7fc700 1 -- 172.21.15.156:0/3058587857 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c022070 con 0x7f584413e340 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f583a7fc700 1 -- 172.21.15.156:0/3058587857 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c00cc90 con 0x7f584413e340 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f584359e700 1 -- 172.21.15.156:0/3058587857 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58440af970 con 0x7f584413e340 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f583a7fc700 1 -- 172.21.15.156:0/3058587857 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c007320 con 0x7f584413e340 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f584359e700 1 -- 172.21.15.156:0/3058587857 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58440ab470 con 0x7f584413e340 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f583a7fc700 1 -- 172.21.15.156:0/3058587857 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f582c007690 con 0x7f584413e340 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f583a7fc700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f583a7fc700 4 mgrc handle_mgr_map Got map version 1 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f583a7fc700 4 mgrc handle_mgr_map Active mgr is now 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f583a7fc700 4 mgrc reconnect No active mgr available yet 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f583a7fc700 1 -- 172.21.15.156:0/3058587857 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f582c01ea60 con 0x7f584413e340 2023-09-29T23:29:19.285 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.281+0000 7f584359e700 1 -- 172.21.15.156:0/3058587857 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58440ab660 con 0x7f584413e340 2023-09-29T23:29:19.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.285+0000 7f583a7fc700 1 -- 172.21.15.156:0/3058587857 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f582c017070 con 0x7f584413e340 2023-09-29T23:29:19.346 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:19.344+0000 7efc73251000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-29T23:29:19.368 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.365+0000 7f584359e700 1 -- 172.21.15.156:0/3058587857 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bf5a122c-0873-4890-8845-702c4fc3325b", "id": 3} v 0) v1 -- 0x7f584405ecb0 con 0x7f584413e340 2023-09-29T23:29:19.375 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.373+0000 7f583a7fc700 1 -- 172.21.15.156:0/3058587857 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bf5a122c-0873-4890-8845-702c4fc3325b", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f582c01ecc0 con 0x7f584413e340 2023-09-29T23:29:19.375 INFO:teuthology.orchestra.run.smithi156.stdout:3 2023-09-29T23:29:19.376 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.373+0000 7f584359e700 10 mgrc shutdown 2023-09-29T23:29:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.373+0000 7f584359e700 1 -- 172.21.15.156:0/3058587857 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f584413e340 msgr2=0x7f584413e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.373+0000 7f584359e700 1 --2- 172.21.15.156:0/3058587857 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f584413e340 0x7f584413e790 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f583000c260 tx=0x7f582c015000 comp rx=0 tx=0).stop 2023-09-29T23:29:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.373+0000 7f584359e700 1 -- 172.21.15.156:0/3058587857 shutdown_connections 2023-09-29T23:29:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.373+0000 7f584359e700 1 --2- 172.21.15.156:0/3058587857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58441504c0 0x7f5844150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.373+0000 7f584359e700 1 --2- 172.21.15.156:0/3058587857 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f584413e340 0x7f584413e790 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.373+0000 7f584359e700 1 --2- 172.21.15.156:0/3058587857 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5830004810 0x7f58440aeda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.373+0000 7f584359e700 1 -- 172.21.15.156:0/3058587857 >> 172.21.15.156:0/3058587857 conn(0x7f584406ab00 msgr2=0x7f58440a57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.373+0000 7f584359e700 1 -- 172.21.15.156:0/3058587857 shutdown_connections 2023-09-29T23:29:19.377 INFO:teuthology.orchestra.run.smithi156.stderr:2023-09-29T23:29:19.373+0000 7f584359e700 1 -- 172.21.15.156:0/3058587857 wait complete. 2023-09-29T23:29:19.380 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:19.377+0000 7f4683d0a000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-29T23:29:19.380 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:19.377+0000 7fc7a7439000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-29T23:29:19.389 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-29T23:29:19.389 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-29T23:29:19.392 INFO:tasks.ceph.osd.0:Started 2023-09-29T23:29:19.392 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-29T23:29:19.392 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-29T23:29:19.394 INFO:tasks.ceph.osd.1:Started 2023-09-29T23:29:19.394 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-29T23:29:19.394 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-29T23:29:19.396 INFO:tasks.ceph.osd.2:Started 2023-09-29T23:29:19.396 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-29T23:29:19.396 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-29T23:29:19.398 INFO:tasks.ceph.osd.3:Started 2023-09-29T23:29:19.399 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T23:29:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.500+0000 7f600dc9f700 1 Processor -- start 2023-09-29T23:29:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.500+0000 7f600dc9f700 1 -- start start 2023-09-29T23:29:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60080b34e0 0x7f600809fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f60080b38b0 0x7f60080a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6008141d70 0x7f60081421e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600809ec70 con 0x7f60080b34e0 2023-09-29T23:29:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60080a0a30 con 0x7f60080b38b0 2023-09-29T23:29:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6008141ab0 con 0x7f6008141d70 2023-09-29T23:29:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600cc9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60080b34e0 0x7f600809fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600cc9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60080b34e0 0x7f600809fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44816/0 (socket says 172.21.15.143:44816) 2023-09-29T23:29:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600cc9d700 1 -- 172.21.15.143:0/276118250 learned_addr learned my addr 172.21.15.143:0/276118250 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600d49e700 1 --2- 172.21.15.143:0/276118250 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6008141d70 0x7f60081421e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f6007fff700 1 --2- 172.21.15.143:0/276118250 >> v2:172.21.15.156:3300/0 conn(0x7f60080b38b0 0x7f60080a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600cc9d700 1 -- 172.21.15.143:0/276118250 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6008141d70 msgr2=0x7f60081421e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600cc9d700 1 --2- 172.21.15.143:0/276118250 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6008141d70 0x7f60081421e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600cc9d700 1 -- 172.21.15.143:0/276118250 >> v2:172.21.15.156:3300/0 conn(0x7f60080b38b0 msgr2=0x7f60080a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600cc9d700 1 --2- 172.21.15.143:0/276118250 >> v2:172.21.15.156:3300/0 conn(0x7f60080b38b0 0x7f60080a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600cc9d700 1 -- 172.21.15.143:0/276118250 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600809ea30 con 0x7f60080b34e0 2023-09-29T23:29:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600cc9d700 1 --2- 172.21.15.143:0/276118250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60080b34e0 0x7f600809fcb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f60080a3490 tx=0x7f5ffc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e419536ee5c423 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f60077fe700 1 -- 172.21.15.143:0/276118250 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc00b640 con 0x7f60080b34e0 2023-09-29T23:29:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f60077fe700 1 -- 172.21.15.143:0/276118250 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc00b7a0 con 0x7f60080b34e0 2023-09-29T23:29:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f60077fe700 1 -- 172.21.15.143:0/276118250 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc00bb60 con 0x7f60080b34e0 2023-09-29T23:29:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 -- 172.21.15.143:0/276118250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60080b34e0 msgr2=0x7f600809fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 --2- 172.21.15.143:0/276118250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60080b34e0 0x7f600809fcb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f60080a3490 tx=0x7f5ffc009250 comp rx=0 tx=0).stop 2023-09-29T23:29:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 -- 172.21.15.143:0/276118250 shutdown_connections 2023-09-29T23:29:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 --2- 172.21.15.143:0/276118250 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6008141d70 0x7f60081421e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 --2- 172.21.15.143:0/276118250 >> v2:172.21.15.156:3300/0 conn(0x7f60080b38b0 0x7f60080a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 --2- 172.21.15.143:0/276118250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60080b34e0 0x7f600809fcb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 -- 172.21.15.143:0/276118250 >> 172.21.15.143:0/276118250 conn(0x7f6008062ed0 msgr2=0x7f60080632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 -- 172.21.15.143:0/276118250 shutdown_connections 2023-09-29T23:29:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 -- 172.21.15.143:0/276118250 wait complete. 2023-09-29T23:29:19.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 Processor -- start 2023-09-29T23:29:19.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.504+0000 7f600dc9f700 1 -- start start 2023-09-29T23:29:19.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600dc9f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60080b34e0 0x7f60080b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600dc9f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60080b38b0 0x7f60080986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600dc9f700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6008141d70 0x7f60080997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600dc9f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600806de80 con 0x7f60080b34e0 2023-09-29T23:29:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600dc9f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60080992e0 con 0x7f60080b38b0 2023-09-29T23:29:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600dc9f700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600809a0e0 con 0x7f6008141d70 2023-09-29T23:29:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600cc9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60080b34e0 0x7f60080b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600cc9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60080b34e0 0x7f60080b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44828/0 (socket says 172.21.15.143:44828) 2023-09-29T23:29:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600cc9d700 1 -- 172.21.15.143:0/4269976530 learned_addr learned my addr 172.21.15.143:0/4269976530 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600d49e700 1 --2- 172.21.15.143:0/4269976530 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6008141d70 0x7f60080997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f6007fff700 1 --2- 172.21.15.143:0/4269976530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60080b38b0 0x7f60080986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600cc9d700 1 -- 172.21.15.143:0/4269976530 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6008141d70 msgr2=0x7f60080997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600cc9d700 1 --2- 172.21.15.143:0/4269976530 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6008141d70 0x7f60080997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600cc9d700 1 -- 172.21.15.143:0/4269976530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60080b38b0 msgr2=0x7f60080986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600cc9d700 1 --2- 172.21.15.143:0/4269976530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60080b38b0 0x7f60080986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600cc9d700 1 -- 172.21.15.143:0/4269976530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc0094f0 con 0x7f60080b34e0 2023-09-29T23:29:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f6007fff700 1 --2- 172.21.15.143:0/4269976530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60080b38b0 0x7f60080986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600cc9d700 1 --2- 172.21.15.143:0/4269976530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60080b34e0 0x7f60080b0a50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6008062d60 tx=0x7f5ffc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f6004ff9700 1 -- 172.21.15.143:0/4269976530 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc000d10 con 0x7f60080b34e0 2023-09-29T23:29:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f6004ff9700 1 -- 172.21.15.143:0/4269976530 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc00bb20 con 0x7f60080b34e0 2023-09-29T23:29:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600dc9f700 1 -- 172.21.15.143:0/4269976530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600806dc40 con 0x7f60080b34e0 2023-09-29T23:29:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f6004ff9700 1 -- 172.21.15.143:0/4269976530 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc011670 con 0x7f60080b34e0 2023-09-29T23:29:19.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600dc9f700 1 -- 172.21.15.143:0/4269976530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600814de00 con 0x7f60080b34e0 2023-09-29T23:29:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f6004ff9700 1 -- 172.21.15.143:0/4269976530 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f5ffc011c60 con 0x7f60080b34e0 2023-09-29T23:29:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f6004ff9700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-09-29T23:29:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f6004ff9700 4 mgrc handle_mgr_map Got map version 1 2023-09-29T23:29:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f6004ff9700 4 mgrc handle_mgr_map Active mgr is now 2023-09-29T23:29:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f6004ff9700 4 mgrc reconnect No active mgr available yet 2023-09-29T23:29:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f6004ff9700 1 -- 172.21.15.143:0/4269976530 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f5ffc02ed60 con 0x7f60080b34e0 2023-09-29T23:29:19.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.508+0000 7f600dc9f700 1 -- 172.21.15.143:0/4269976530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600805ecb0 con 0x7f60080b34e0 2023-09-29T23:29:19.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.512+0000 7f6004ff9700 1 -- 172.21.15.143:0/4269976530 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f5ffc033060 con 0x7f60080b34e0 2023-09-29T23:29:19.577 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:19.576+0000 7efc73251000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-29T23:29:19.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.576+0000 7f600dc9f700 1 -- 172.21.15.143:0/4269976530 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f600809cce0 con 0x7f60080b34e0 2023-09-29T23:29:19.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.580+0000 7f6004ff9700 1 -- 172.21.15.143:0/4269976530 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7f5ffc015020 con 0x7f60080b34e0 2023-09-29T23:29:19.582 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:19.582 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":6,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:29:19.370491+0000","last_up_change":"0.000000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:29:19.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.580+0000 7f600dc9f700 10 mgrc shutdown 2023-09-29T23:29:19.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.580+0000 7f600dc9f700 1 -- 172.21.15.143:0/4269976530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60080b34e0 msgr2=0x7f60080b0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:19.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.580+0000 7f600dc9f700 1 --2- 172.21.15.143:0/4269976530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60080b34e0 0x7f60080b0a50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6008062d60 tx=0x7f5ffc017000 comp rx=0 tx=0).stop 2023-09-29T23:29:19.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.580+0000 7f600dc9f700 1 -- 172.21.15.143:0/4269976530 shutdown_connections 2023-09-29T23:29:19.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.580+0000 7f600dc9f700 1 --2- 172.21.15.143:0/4269976530 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6008141d70 0x7f60080997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.580+0000 7f600dc9f700 1 --2- 172.21.15.143:0/4269976530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60080b38b0 0x7f60080986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.580+0000 7f600dc9f700 1 --2- 172.21.15.143:0/4269976530 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60080b34e0 0x7f60080b0a50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:19.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.580+0000 7f600dc9f700 1 -- 172.21.15.143:0/4269976530 >> 172.21.15.143:0/4269976530 conn(0x7f6008062ed0 msgr2=0x7f60080684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:19.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.580+0000 7f600dc9f700 1 -- 172.21.15.143:0/4269976530 shutdown_connections 2023-09-29T23:29:19.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:19.580+0000 7f600dc9f700 1 -- 172.21.15.143:0/4269976530 wait complete. 2023-09-29T23:29:19.594 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-09-29T23:29:19.594 INFO:tasks.ceph:Waiting for OSDs to come up 2023-09-29T23:29:19.623 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:19.621+0000 7fc7a7439000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-29T23:29:19.624 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:19.621+0000 7f4683d0a000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-29T23:29:19.627 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:19.624+0000 7efc73251000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-29T23:29:19.671 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:19.669+0000 7f4683d0a000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-29T23:29:19.672 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:19.669+0000 7fc7a7439000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-29T23:29:19.775 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-09-29T23:29:19.773+0000 7f6731fed3c0 -1 Falling back to public interface 2023-09-29T23:29:19.811 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:19.808+0000 7efc73251000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-29T23:29:19.867 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:19.865+0000 7fc7a7439000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-29T23:29:19.867 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:19.865+0000 7f4683d0a000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-29T23:29:19.873 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:19.872+0000 7efc73251000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-29T23:29:19.928 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:19.925+0000 7f4683d0a000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-29T23:29:19.929 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:19.925+0000 7fc7a7439000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-29T23:29:19.933 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:19.932+0000 7efc73251000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-29T23:29:19.991 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:19.989+0000 7f4683d0a000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-29T23:29:19.991 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:19.989+0000 7fc7a7439000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-29T23:29:19.997 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-29T23:29:20.018 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-09-29T23:29:20.017+0000 7fb85ac453c0 -1 Falling back to public interface 2023-09-29T23:29:20.019 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T23:29:20.016+0000 7fc4e49573c0 -1 Falling back to public interface 2023-09-29T23:29:20.047 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-09-29T23:29:20.045+0000 7fa7063803c0 -1 Falling back to public interface 2023-09-29T23:29:20.084 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.080+0000 7fc837b41700 1 Processor -- start 2023-09-29T23:29:20.084 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.080+0000 7fc837b41700 1 -- start start 2023-09-29T23:29:20.087 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc8300b7080 0x7fc83006bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:20.087 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc83006c450 0x7fc830135f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:20.088 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc8301364a0 0x7fc8300b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:20.088 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8300b44b0 con 0x7fc8300b7080 2023-09-29T23:29:20.088 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc83006b730 con 0x7fc83006c450 2023-09-29T23:29:20.088 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300b7450 con 0x7fc8301364a0 2023-09-29T23:29:20.088 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8358dd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc8300b7080 0x7fc83006bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:20.088 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8360de700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc8301364a0 0x7fc8300b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:20.089 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8358dd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc8300b7080 0x7fc83006bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:44838/0 (socket says 172.21.15.143:44838) 2023-09-29T23:29:20.089 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8358dd700 1 -- 172.21.15.143:0/3128806967 learned_addr learned my addr 172.21.15.143:0/3128806967 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:20.089 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8350dc700 1 --2- 172.21.15.143:0/3128806967 >> v2:172.21.15.156:3300/0 conn(0x7fc83006c450 0x7fc830135f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:20.089 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8358dd700 1 -- 172.21.15.143:0/3128806967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc8301364a0 msgr2=0x7fc8300b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:20.089 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8358dd700 1 --2- 172.21.15.143:0/3128806967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc8301364a0 0x7fc8300b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:20.090 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8358dd700 1 -- 172.21.15.143:0/3128806967 >> v2:172.21.15.156:3300/0 conn(0x7fc83006c450 msgr2=0x7fc830135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:20.090 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8358dd700 1 --2- 172.21.15.143:0/3128806967 >> v2:172.21.15.156:3300/0 conn(0x7fc83006c450 0x7fc830135f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:20.090 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8358dd700 1 -- 172.21.15.143:0/3128806967 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8300b41f0 con 0x7fc8300b7080 2023-09-29T23:29:20.090 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8358dd700 1 --2- 172.21.15.143:0/3128806967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8300b7080 0x7fc83006bf10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc83006e230 tx=0x7fc828009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a1ddc049440573 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:20.090 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8348db700 1 -- 172.21.15.143:0/3128806967 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800e070 con 0x7fc8300b7080 2023-09-29T23:29:20.091 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8348db700 1 -- 172.21.15.143:0/3128806967 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82800ba20 con 0x7fc8300b7080 2023-09-29T23:29:20.091 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc8348db700 1 -- 172.21.15.143:0/3128806967 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8280092e0 con 0x7fc8300b7080 2023-09-29T23:29:20.091 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 -- 172.21.15.143:0/3128806967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8300b7080 msgr2=0x7fc83006bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:20.091 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 --2- 172.21.15.143:0/3128806967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8300b7080 0x7fc83006bf10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc83006e230 tx=0x7fc828009840 comp rx=0 tx=0).stop 2023-09-29T23:29:20.091 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 -- 172.21.15.143:0/3128806967 shutdown_connections 2023-09-29T23:29:20.092 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 --2- 172.21.15.143:0/3128806967 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc8301364a0 0x7fc8300b3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:20.092 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 --2- 172.21.15.143:0/3128806967 >> v2:172.21.15.156:3300/0 conn(0x7fc83006c450 0x7fc830135f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:20.092 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 --2- 172.21.15.143:0/3128806967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8300b7080 0x7fc83006bf10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:20.092 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 -- 172.21.15.143:0/3128806967 >> 172.21.15.143:0/3128806967 conn(0x7fc830069d80 msgr2=0x7fc83006a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:20.092 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 -- 172.21.15.143:0/3128806967 shutdown_connections 2023-09-29T23:29:20.092 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 -- 172.21.15.143:0/3128806967 wait complete. 2023-09-29T23:29:20.096 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 Processor -- start 2023-09-29T23:29:20.096 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.084+0000 7fc837b41700 1 -- start start 2023-09-29T23:29:20.096 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc837b41700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc83006c450 0x7fc8300af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:20.096 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc837b41700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8300b7080 0x7fc8300aaa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:20.096 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc837b41700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8301364a0 0x7fc8300aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:20.096 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc837b41700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300ab6f0 con 0x7fc8301364a0 2023-09-29T23:29:20.097 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc837b41700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8300ab9b0 con 0x7fc8300b7080 2023-09-29T23:29:20.097 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc837b41700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc830133b20 con 0x7fc83006c450 2023-09-29T23:29:20.097 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8358dd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc83006c450 0x7fc8300af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:20.097 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8358dd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc83006c450 0x7fc8300af580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:49998/0 (socket says 172.21.15.143:49998) 2023-09-29T23:29:20.097 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8360de700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8301364a0 0x7fc8300aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:20.097 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8358dd700 1 -- 172.21.15.143:0/111711807 learned_addr learned my addr 172.21.15.143:0/111711807 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:20.097 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8350dc700 1 --2- 172.21.15.143:0/111711807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8300b7080 0x7fc8300aaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:20.097 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8360de700 1 -- 172.21.15.143:0/111711807 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc83006c450 msgr2=0x7fc8300af580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:20.097 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8360de700 1 --2- 172.21.15.143:0/111711807 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc83006c450 0x7fc8300af580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:20.097 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8360de700 1 -- 172.21.15.143:0/111711807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8300b7080 msgr2=0x7fc8300aaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:20.098 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8360de700 1 --2- 172.21.15.143:0/111711807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8300b7080 0x7fc8300aaa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:20.098 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8360de700 1 -- 172.21.15.143:0/111711807 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc828009ae0 con 0x7fc8301364a0 2023-09-29T23:29:20.098 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8358dd700 1 --2- 172.21.15.143:0/111711807 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc83006c450 0x7fc8300af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:20.098 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc8360de700 1 --2- 172.21.15.143:0/111711807 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8301364a0 0x7fc8300aafc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00e8d0 tx=0x7fc82c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:20.098 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc825ffb700 1 -- 172.21.15.143:0/111711807 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c013680 con 0x7fc8301364a0 2023-09-29T23:29:20.098 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc825ffb700 1 -- 172.21.15.143:0/111711807 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82c0137e0 con 0x7fc8301364a0 2023-09-29T23:29:20.098 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc837b41700 1 -- 172.21.15.143:0/111711807 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8301335e0 con 0x7fc8301364a0 2023-09-29T23:29:20.098 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc825ffb700 1 -- 172.21.15.143:0/111711807 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c013ba0 con 0x7fc8301364a0 2023-09-29T23:29:20.099 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc837b41700 1 -- 172.21.15.143:0/111711807 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8300afc60 con 0x7fc8301364a0 2023-09-29T23:29:20.099 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc825ffb700 1 -- 172.21.15.143:0/111711807 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc82c005c80 con 0x7fc8301364a0 2023-09-29T23:29:20.099 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc825ffb700 20 mgrc handle_mgr_map mgrmap(e 1) v1 2023-09-29T23:29:20.099 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc825ffb700 4 mgrc handle_mgr_map Got map version 1 2023-09-29T23:29:20.099 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc825ffb700 4 mgrc handle_mgr_map Active mgr is now 2023-09-29T23:29:20.099 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc825ffb700 4 mgrc reconnect No active mgr available yet 2023-09-29T23:29:20.099 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc825ffb700 1 -- 172.21.15.143:0/111711807 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fc82c005f30 con 0x7fc8301364a0 2023-09-29T23:29:20.099 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.088+0000 7fc837b41700 1 -- 172.21.15.143:0/111711807 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83005ecb0 con 0x7fc8301364a0 2023-09-29T23:29:20.099 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.092+0000 7fc825ffb700 1 -- 172.21.15.143:0/111711807 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc82c021060 con 0x7fc8301364a0 2023-09-29T23:29:20.155 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.152+0000 7fc837b41700 1 -- 172.21.15.143:0/111711807 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8300b3aa0 con 0x7fc8301364a0 2023-09-29T23:29:20.156 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.152+0000 7fc825ffb700 1 -- 172.21.15.143:0/111711807 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4108 (secure 0 0 0) 0x7fc82c017070 con 0x7fc8301364a0 2023-09-29T23:29:20.156 INFO:teuthology.misc.health.smithi143.stdout: 2023-09-29T23:29:20.156 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":6,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:29:19.370491+0000","last_up_change":"0.000000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:29:20.157 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.152+0000 7fc837b41700 10 mgrc shutdown 2023-09-29T23:29:20.158 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.152+0000 7fc837b41700 1 -- 172.21.15.143:0/111711807 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8301364a0 msgr2=0x7fc8300aafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:20.158 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.152+0000 7fc837b41700 1 --2- 172.21.15.143:0/111711807 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8301364a0 0x7fc8300aafc0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00e8d0 tx=0x7fc82c0091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:20.158 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.152+0000 7fc837b41700 1 -- 172.21.15.143:0/111711807 shutdown_connections 2023-09-29T23:29:20.158 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.152+0000 7fc837b41700 1 --2- 172.21.15.143:0/111711807 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc8301364a0 0x7fc8300aafc0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:20.158 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.152+0000 7fc837b41700 1 --2- 172.21.15.143:0/111711807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8300b7080 0x7fc8300aaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:20.158 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.152+0000 7fc837b41700 1 --2- 172.21.15.143:0/111711807 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc83006c450 0x7fc8300af580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:20.158 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.152+0000 7fc837b41700 1 -- 172.21.15.143:0/111711807 >> 172.21.15.143:0/111711807 conn(0x7fc830069d80 msgr2=0x7fc8300b2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:20.158 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.156+0000 7fc837b41700 1 -- 172.21.15.143:0/111711807 shutdown_connections 2023-09-29T23:29:20.158 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:20.156+0000 7fc837b41700 1 -- 172.21.15.143:0/111711807 wait complete. 2023-09-29T23:29:20.166 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-09-29T23:29:20.337 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:20.336+0000 7efc73251000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-29T23:29:20.441 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:20.437+0000 7fc7a7439000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-29T23:29:20.441 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:20.437+0000 7f4683d0a000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-29T23:29:20.458 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:20.456+0000 7efc73251000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-29T23:29:20.557 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:20.553+0000 7f4683d0a000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-29T23:29:20.557 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:20.553+0000 7fc7a7439000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-29T23:29:20.768 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:20.764+0000 7efc73251000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-29T23:29:20.857 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:20.853+0000 7f4683d0a000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-29T23:29:20.859 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:20.857+0000 7fc7a7439000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-29T23:29:20.930 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:20.928+0000 7efc73251000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-29T23:29:20.993 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:20.992+0000 7efc73251000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-29T23:29:21.021 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:21.017+0000 7fc7a7439000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-29T23:29:21.025 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:21.021+0000 7f4683d0a000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-29T23:29:21.045 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:21.044+0000 7efc73251000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-29T23:29:21.089 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:21.085+0000 7fc7a7439000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-29T23:29:21.091 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:21.089+0000 7f4683d0a000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-29T23:29:21.102 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:21.100+0000 7efc73251000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-29T23:29:21.144 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:21.141+0000 7f4683d0a000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-29T23:29:21.144 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:21.141+0000 7fc7a7439000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-29T23:29:21.203 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:21.201+0000 7f4683d0a000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-29T23:29:21.204 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:21.201+0000 7fc7a7439000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-29T23:29:21.391 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:21.388+0000 7efc73251000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-29T23:29:21.446 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:21.444+0000 7efc73251000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-29T23:29:21.495 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:21.493+0000 7fc7a7439000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-29T23:29:21.495 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:21.493+0000 7f4683d0a000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-29T23:29:21.509 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:21.508+0000 7efc73251000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-29T23:29:21.548 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:21.545+0000 7f4683d0a000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-29T23:29:21.548 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:21.545+0000 7fc7a7439000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-29T23:29:21.609 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:21.605+0000 7f4683d0a000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-29T23:29:21.614 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:21.613+0000 7fc7a7439000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-29T23:29:21.665 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:21.664+0000 7efc73251000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-29T23:29:21.730 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:21.728+0000 7efc73251000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-29T23:29:21.772 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:21.769+0000 7f4683d0a000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-29T23:29:21.776 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:21.773+0000 7fc7a7439000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-29T23:29:21.834 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:21.833+0000 7f4683d0a000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-29T23:29:21.836 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:21.833+0000 7fc7a7439000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-29T23:29:22.426 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:22.424+0000 7efc73251000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-29T23:29:22.478 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:22.476+0000 7efc50b36700 -1 log_channel(cluster) log [ERR] : Unhandled exception from module 'pg_autoscaler' while running on mgr.x: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-29T23:29:22.478 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:22.476+0000 7efc50b36700 -1 pg_autoscaler.serve: 2023-09-29T23:29:22.480 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:22.476+0000 7efc50b36700 -1 Traceback (most recent call last): 2023-09-29T23:29:22.480 INFO:tasks.ceph.mgr.x.smithi143.stderr: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 325, in serve 2023-09-29T23:29:22.480 INFO:tasks.ceph.mgr.x.smithi143.stderr: self._maybe_adjust() 2023-09-29T23:29:22.480 INFO:tasks.ceph.mgr.x.smithi143.stderr: File "/usr/share/ceph/mgr/pg_autoscaler/module.py", line 682, in _maybe_adjust 2023-09-29T23:29:22.480 INFO:tasks.ceph.mgr.x.smithi143.stderr: if self.noautoscale: 2023-09-29T23:29:22.480 INFO:tasks.ceph.mgr.x.smithi143.stderr:AttributeError: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-29T23:29:22.480 INFO:tasks.ceph.mgr.x.smithi143.stderr: 2023-09-29T23:29:22.558 INFO:tasks.ceph.mgr.y.smithi156.stderr:2023-09-29T23:29:22.557+0000 7f4683d0a000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-29T23:29:22.558 INFO:tasks.ceph.mgr.z.smithi156.stderr:2023-09-29T23:29:22.557+0000 7fc7a7439000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-29T23:29:23.473 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:23.468+0000 7efc5c34d700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-29T23:29:23.473 INFO:tasks.ceph.mgr.x.smithi143.stderr:2023-09-29T23:29:23.472+0000 7efc5c34d700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-29T23:29:23.664 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-09-29T23:29:23.661+0000 7f6731fed3c0 -1 osd.3 0 log_to_monitors true 2023-09-29T23:29:23.771 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T23:29:23.768+0000 7fc4e49573c0 -1 osd.0 0 log_to_monitors true 2023-09-29T23:29:23.837 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-09-29T23:29:23.833+0000 7fb85ac453c0 -1 osd.2 0 log_to_monitors true 2023-09-29T23:29:23.911 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-09-29T23:29:23.909+0000 7fa7063803c0 -1 osd.1 0 log_to_monitors true 2023-09-29T23:29:25.801 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:29:25.803 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:29:26.336 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T23:29:26.332+0000 7fc4e191c700 -1 osd.0 0 waiting for initial osdmap 2023-09-29T23:29:26.342 INFO:tasks.ceph.osd.0.smithi143.stderr:2023-09-29T23:29:26.340+0000 7fc4d8e9b700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-29T23:29:26.479 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-09-29T23:29:26.477+0000 7fa703345700 -1 osd.1 0 waiting for initial osdmap 2023-09-29T23:29:26.485 INFO:tasks.ceph.osd.1.smithi156.stderr:2023-09-29T23:29:26.481+0000 7fa6fa8c4700 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-29T23:29:26.490 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-09-29T23:29:26.489+0000 7fb8563e5700 -1 osd.2 0 waiting for initial osdmap 2023-09-29T23:29:26.494 INFO:tasks.ceph.osd.2.smithi156.stderr:2023-09-29T23:29:26.493+0000 7fb84f189700 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-29T23:29:26.497 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-09-29T23:29:26.493+0000 7f672d78d700 -1 osd.3 0 waiting for initial osdmap 2023-09-29T23:29:26.500 INFO:tasks.ceph.osd.3.smithi156.stderr:2023-09-29T23:29:26.497+0000 7f6726531700 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-29T23:29:26.570 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-29T23:29:26.672 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.668+0000 7fb5d975e700 1 Processor -- start 2023-09-29T23:29:26.672 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.668+0000 7fb5d975e700 1 -- start start 2023-09-29T23:29:26.673 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.668+0000 7fb5d975e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb5d40a2060 0x7fb5d41482c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:26.673 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.668+0000 7fb5d975e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d406a1d0 con 0x7fb5d406e890 2023-09-29T23:29:26.673 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.668+0000 7fb5d975e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d406c320 con 0x7fb5d4069e00 2023-09-29T23:29:26.673 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.668+0000 7fb5d975e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5d406bc80 con 0x7fb5d40a2060 2023-09-29T23:29:26.673 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5cb7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb5d406e890 0x7fb5d40a1950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34744/0 (socket says 172.21.15.143:34744) 2023-09-29T23:29:26.673 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d8f5d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb5d40a2060 0x7fb5d41482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:26.673 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5cb7fe700 1 -- 172.21.15.143:0/3617446628 learned_addr learned my addr 172.21.15.143:0/3617446628 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:26.674 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d37fe700 1 -- 172.21.15.143:0/3617446628 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4134704807 0 0) 0x7fb5d406a1d0 con 0x7fb5d406e890 2023-09-29T23:29:26.675 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d37fe700 1 -- 172.21.15.143:0/3617446628 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b4003570 con 0x7fb5d406e890 2023-09-29T23:29:26.675 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d8f5d700 1 -- 172.21.15.143:0/3617446628 >> v1:172.21.15.156:6789/0 conn(0x7fb5d4069e00 legacy=0x7fb5d413d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:26.675 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d8f5d700 1 -- 172.21.15.143:0/3617446628 >> v1:172.21.15.143:6789/0 conn(0x7fb5d406e890 legacy=0x7fb5d40a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:26.675 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d8f5d700 1 -- 172.21.15.143:0/3617446628 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d406c5e0 con 0x7fb5d40a2060 2023-09-29T23:29:26.675 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d8f5d700 1 --2- 172.21.15.143:0/3617446628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d40a2060 0x7fb5d41482c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009e60 tx=0x7fb5cc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d793a32edd33b37 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:26.675 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d37fe700 1 -- 172.21.15.143:0/3617446628 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc00e070 con 0x7fb5d40a2060 2023-09-29T23:29:26.675 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d37fe700 1 -- 172.21.15.143:0/3617446628 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc00bb60 con 0x7fb5d40a2060 2023-09-29T23:29:26.675 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d37fe700 1 -- 172.21.15.143:0/3617446628 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc012430 con 0x7fb5d40a2060 2023-09-29T23:29:26.678 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d975e700 1 -- 172.21.15.143:0/3617446628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d40a2060 msgr2=0x7fb5d41482c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:26.679 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d975e700 1 --2- 172.21.15.143:0/3617446628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d40a2060 0x7fb5d41482c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009e60 tx=0x7fb5cc009780 comp rx=0 tx=0).stop 2023-09-29T23:29:26.679 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d975e700 1 -- 172.21.15.143:0/3617446628 shutdown_connections 2023-09-29T23:29:26.679 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d975e700 1 --2- 172.21.15.143:0/3617446628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d40a2060 0x7fb5d41482c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:26.679 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d975e700 1 -- 172.21.15.143:0/3617446628 >> 172.21.15.143:0/3617446628 conn(0x7fb5d406fc90 msgr2=0x7fb5d4070090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:26.679 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d975e700 1 -- 172.21.15.143:0/3617446628 shutdown_connections 2023-09-29T23:29:26.679 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.672+0000 7fb5d975e700 1 -- 172.21.15.143:0/3617446628 wait complete. 2023-09-29T23:29:26.679 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d975e700 1 Processor -- start 2023-09-29T23:29:26.679 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d975e700 1 -- start start 2023-09-29T23:29:26.679 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d975e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d4069e00 0x7fb5d409bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:26.679 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d975e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d406e890 0x7fb5d4139170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:26.680 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d975e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb5d40a2060 0x7fb5d409a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:26.680 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d975e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d409b830 con 0x7fb5d406e890 2023-09-29T23:29:26.680 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d975e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d4139b40 con 0x7fb5d4069e00 2023-09-29T23:29:26.680 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d975e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d4139de0 con 0x7fb5d40a2060 2023-09-29T23:29:26.680 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5cb7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d406e890 0x7fb5d4139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:26.680 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5cb7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d406e890 0x7fb5d4139170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:40604/0 (socket says 172.21.15.143:40604) 2023-09-29T23:29:26.680 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5cb7fe700 1 -- 172.21.15.143:0/3830233274 learned_addr learned my addr 172.21.15.143:0/3830233274 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:26.680 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d8f5d700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb5d40a2060 0x7fb5d409a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:26.680 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d3fff700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d4069e00 0x7fb5d409bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:26.681 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5cb7fe700 1 -- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb5d40a2060 msgr2=0x7fb5d409a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:26.682 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5cb7fe700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb5d40a2060 0x7fb5d409a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:26.682 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5cb7fe700 1 -- 172.21.15.143:0/3830233274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d4069e00 msgr2=0x7fb5d409bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:26.682 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5cb7fe700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d4069e00 0x7fb5d409bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:26.682 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5cb7fe700 1 -- 172.21.15.143:0/3830233274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc009a20 con 0x7fb5d406e890 2023-09-29T23:29:26.682 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d3fff700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d4069e00 0x7fb5d409bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:26.682 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5cb7fe700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d406e890 0x7fb5d4139170 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00c7c0 tx=0x7fb5bc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:26.682 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d0ff9700 1 -- 172.21.15.143:0/3830233274 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc005b30 con 0x7fb5d406e890 2023-09-29T23:29:26.682 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d0ff9700 1 -- 172.21.15.143:0/3830233274 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc005cb0 con 0x7fb5d406e890 2023-09-29T23:29:26.682 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d975e700 1 -- 172.21.15.143:0/3830233274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d409c690 con 0x7fb5d406e890 2023-09-29T23:29:26.682 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d0ff9700 1 -- 172.21.15.143:0/3830233274 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc007320 con 0x7fb5d406e890 2023-09-29T23:29:26.682 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.676+0000 7fb5d975e700 1 -- 172.21.15.143:0/3830233274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d4098790 con 0x7fb5d406e890 2023-09-29T23:29:26.683 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.680+0000 7fb5d975e700 1 -- 172.21.15.143:0/3830233274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d405ecb0 con 0x7fb5d406e890 2023-09-29T23:29:26.683 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.680+0000 7fb5d0ff9700 1 -- 172.21.15.143:0/3830233274 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 4) v1 ==== 119683+0+0 (secure 0 0 0) 0x7fb5bc01b3b0 con 0x7fb5d406e890 2023-09-29T23:29:26.683 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.680+0000 7fb5d0ff9700 20 mgrc handle_mgr_map mgrmap(e 4) v1 2023-09-29T23:29:26.684 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.680+0000 7fb5d0ff9700 4 mgrc handle_mgr_map Got map version 4 2023-09-29T23:29:26.684 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.680+0000 7fb5d0ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:26.684 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.680+0000 7fb5d0ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:26.684 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.680+0000 7fb5d0ff9700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb5b808e820 0x7fb5b8090cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:26.684 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.680+0000 7fb5d0ff9700 1 -- 172.21.15.143:0/3830233274 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1619+0+0 (secure 0 0 0) 0x7fb5bc0b3640 con 0x7fb5d406e890 2023-09-29T23:29:26.684 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.680+0000 7fb5d3fff700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb5b808e820 0x7fb5b8090cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:26.684 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.680+0000 7fb5d3fff700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb5b808e820 0x7fb5b8090cd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4007fd0 tx=0x7fb5c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:26.691 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.688+0000 7fb5d0ff9700 1 -- 172.21.15.143:0/3830233274 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb5bc084b40 con 0x7fb5d406e890 2023-09-29T23:29:26.851 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.848+0000 7fb5d975e700 1 -- 172.21.15.143:0/3830233274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5d409b970 con 0x7fb5d406e890 2023-09-29T23:29:26.852 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.848+0000 7fb5d0ff9700 1 -- 172.21.15.143:0/3830233274 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v8) v1 ==== 74+0+4108 (secure 0 0 0) 0x7fb5d409b970 con 0x7fb5d406e890 2023-09-29T23:29:26.852 INFO:teuthology.misc.health.smithi143.stdout: 2023-09-29T23:29:26.852 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":8,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:29:25.463834+0000","last_up_change":"0.000000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:29:26.855 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 10 mgrc shutdown 2023-09-29T23:29:26.855 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 -- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb5b808e820 msgr2=0x7fb5b8090cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:26.855 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb5b808e820 0x7fb5b8090cd0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb5c4007fd0 tx=0x7fb5c4009040 comp rx=0 tx=0).stop 2023-09-29T23:29:26.855 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 -- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d406e890 msgr2=0x7fb5d4139170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:26.855 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d406e890 0x7fb5d4139170 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc00c7c0 tx=0x7fb5bc007e00 comp rx=0 tx=0).stop 2023-09-29T23:29:26.855 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 -- 172.21.15.143:0/3830233274 shutdown_connections 2023-09-29T23:29:26.855 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb5b808e820 0x7fb5b8090cd0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:26.855 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb5d40a2060 0x7fb5d409a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:26.855 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5d406e890 0x7fb5d4139170 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:26.856 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 --2- 172.21.15.143:0/3830233274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d4069e00 0x7fb5d409bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:26.856 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 -- 172.21.15.143:0/3830233274 >> 172.21.15.143:0/3830233274 conn(0x7fb5d406fc90 msgr2=0x7fb5d406eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:26.856 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 -- 172.21.15.143:0/3830233274 shutdown_connections 2023-09-29T23:29:26.856 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:26.852+0000 7fb5d975e700 1 -- 172.21.15.143:0/3830233274 wait complete. 2023-09-29T23:29:26.865 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-09-29T23:29:27.505 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-09-29T23:29:27.500+0000 7fdae1904700 -1 log_channel(cluster) log [ERR] : Health check failed: Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale' (MGR_MODULE_ERROR) 2023-09-29T23:29:33.268 DEBUG:teuthology.orchestra.run.smithi143:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-29T23:29:33.371 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744d699700 1 Processor -- start 2023-09-29T23:29:33.371 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744d699700 1 -- start start 2023-09-29T23:29:33.372 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744d699700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7448069d30 0x7f744806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:33.373 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744d699700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744813f980 0x7f7448140030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:33.373 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744d699700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744806a100 con 0x7f744806e910 2023-09-29T23:29:33.373 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744d699700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74480b7800 con 0x7f7448069d30 2023-09-29T23:29:33.373 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744d699700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74480a63e0 con 0x7f744813f980 2023-09-29T23:29:33.373 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f74477fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f744806e910 0x7f744813f270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54696/0 (socket says 172.21.15.143:54696) 2023-09-29T23:29:33.373 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f74477fe700 1 -- 172.21.15.143:0/2927034103 learned_addr learned my addr 172.21.15.143:0/2927034103 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:33.373 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744ce98700 1 --2- 172.21.15.143:0/2927034103 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744813f980 0x7f7448140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:33.373 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f7447fff700 1 --2- 172.21.15.143:0/2927034103 >> v2:172.21.15.156:3300/0 conn(0x7f7448069d30 0x7f744806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:33.373 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f7446ffd700 1 -- 172.21.15.143:0/2927034103 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223737844 0 0) 0x7f744806a100 con 0x7f744806e910 2023-09-29T23:29:33.373 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744ce98700 1 -- 172.21.15.143:0/2927034103 >> v2:172.21.15.156:3300/0 conn(0x7f7448069d30 msgr2=0x7f744806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:33.373 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744ce98700 1 --2- 172.21.15.143:0/2927034103 >> v2:172.21.15.156:3300/0 conn(0x7f7448069d30 0x7f744806b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.374 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744ce98700 1 -- 172.21.15.143:0/2927034103 >> v1:172.21.15.143:6789/0 conn(0x7f744806e910 legacy=0x7f744813f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:33.374 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744ce98700 1 -- 172.21.15.143:0/2927034103 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744806ba20 con 0x7f744813f980 2023-09-29T23:29:33.374 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f7447fff700 1 --2- 172.21.15.143:0/2927034103 >> v2:172.21.15.156:3300/0 conn(0x7f7448069d30 0x7f744806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:33.374 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.368+0000 7f744ce98700 1 --2- 172.21.15.143:0/2927034103 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744813f980 0x7f7448140030 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7440009910 tx=0x7f74400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f17bc0c62178521 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:33.375 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f7446ffd700 1 -- 172.21.15.143:0/2927034103 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744000b820 con 0x7f744813f980 2023-09-29T23:29:33.375 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f7446ffd700 1 -- 172.21.15.143:0/2927034103 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744000b980 con 0x7f744813f980 2023-09-29T23:29:33.375 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f7446ffd700 1 -- 172.21.15.143:0/2927034103 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744000bd20 con 0x7f744813f980 2023-09-29T23:29:33.375 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 -- 172.21.15.143:0/2927034103 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744813f980 msgr2=0x7f7448140030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:33.375 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 --2- 172.21.15.143:0/2927034103 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744813f980 0x7f7448140030 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7440009910 tx=0x7f74400091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.376 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 -- 172.21.15.143:0/2927034103 shutdown_connections 2023-09-29T23:29:33.376 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 --2- 172.21.15.143:0/2927034103 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744813f980 0x7f7448140030 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.376 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 --2- 172.21.15.143:0/2927034103 >> v2:172.21.15.156:3300/0 conn(0x7f7448069d30 0x7f744806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.376 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 -- 172.21.15.143:0/2927034103 >> 172.21.15.143:0/2927034103 conn(0x7f74480677f0 msgr2=0x7f744806fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:33.376 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 -- 172.21.15.143:0/2927034103 shutdown_connections 2023-09-29T23:29:33.376 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 -- 172.21.15.143:0/2927034103 wait complete. 2023-09-29T23:29:33.376 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 Processor -- start 2023-09-29T23:29:33.376 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 -- start start 2023-09-29T23:29:33.378 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7448069d30 0x7f744809cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:33.378 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744806e910 0x7f744809d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:33.378 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f744813f980 0x7f744813e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:33.378 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744809c390 con 0x7f7448069d30 2023-09-29T23:29:33.378 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448143720 con 0x7f744813f980 2023-09-29T23:29:33.378 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744d699700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7448143460 con 0x7f744806e910 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f7447fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7448069d30 0x7f744809cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f74477fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744806e910 0x7f744809d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f7447fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7448069d30 0x7f744809cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47662/0 (socket says 172.21.15.143:47662) 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f7447fff700 1 -- 172.21.15.143:0/967964908 learned_addr learned my addr 172.21.15.143:0/967964908 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f74477fe700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744806e910 0x7f744809d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60150/0 (socket says 172.21.15.143:60150) 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f744ce98700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f744813f980 0x7f744813e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f7447fff700 1 -- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744806e910 msgr2=0x7f744809d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.372+0000 7f7447fff700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744806e910 0x7f744809d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.376+0000 7f7447fff700 1 -- 172.21.15.143:0/967964908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f744813f980 msgr2=0x7f744813e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.376+0000 7f7447fff700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f744813f980 0x7f744813e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.376+0000 7f7447fff700 1 -- 172.21.15.143:0/967964908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7440009420 con 0x7f7448069d30 2023-09-29T23:29:33.379 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.376+0000 7f74477fe700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744806e910 0x7f744809d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:33.380 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.376+0000 7f7447fff700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7448069d30 0x7f744809cf70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f743000e8d0 tx=0x7f7430008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:33.380 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.376+0000 7f743ffff700 1 -- 172.21.15.143:0/967964908 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430013410 con 0x7f7448069d30 2023-09-29T23:29:33.380 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.376+0000 7f743ffff700 1 -- 172.21.15.143:0/967964908 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7430013570 con 0x7f7448069d30 2023-09-29T23:29:33.380 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.376+0000 7f744d699700 1 -- 172.21.15.143:0/967964908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7448143c00 con 0x7f7448069d30 2023-09-29T23:29:33.380 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.376+0000 7f743ffff700 1 -- 172.21.15.143:0/967964908 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430013910 con 0x7f7448069d30 2023-09-29T23:29:33.380 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.376+0000 7f744d699700 1 -- 172.21.15.143:0/967964908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744813edd0 con 0x7f7448069d30 2023-09-29T23:29:33.380 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.376+0000 7f744d699700 1 -- 172.21.15.143:0/967964908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744805ecb0 con 0x7f7448069d30 2023-09-29T23:29:33.381 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.380+0000 7f743ffff700 1 -- 172.21.15.143:0/967964908 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f7430013d40 con 0x7f7448069d30 2023-09-29T23:29:33.381 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.380+0000 7f743ffff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:33.382 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.380+0000 7f743ffff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:33.382 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.380+0000 7f743ffff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:33.382 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.380+0000 7f743ffff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:33.382 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.380+0000 7f743ffff700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f742808e8e0 0x7f7428090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:33.382 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.380+0000 7f743ffff700 1 -- 172.21.15.143:0/967964908 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 3125+0+0 (secure 0 0 0) 0x7f74300b4350 con 0x7f7448069d30 2023-09-29T23:29:33.382 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.380+0000 7f74477fe700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f742808e8e0 0x7f7428090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:33.383 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.380+0000 7f74477fe700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f742808e8e0 0x7f7428090d90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f744806d790 tx=0x7f7438001a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:33.390 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.388+0000 7f743ffff700 1 -- 172.21.15.143:0/967964908 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f74300852c0 con 0x7f7448069d30 2023-09-29T23:29:33.554 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.552+0000 7f744d699700 1 -- 172.21.15.143:0/967964908 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7448140570 con 0x7f7448069d30 2023-09-29T23:29:33.555 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.552+0000 7f743ffff700 1 -- 172.21.15.143:0/967964908 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7471 (secure 0 0 0) 0x7f74300888e0 con 0x7f7448069d30 2023-09-29T23:29:33.556 INFO:teuthology.misc.health.smithi143.stdout: 2023-09-29T23:29:33.556 INFO:teuthology.misc.health.smithi143.stdout:{"epoch":12,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:29:30.535250+0000","last_up_change":"2023-09-29T23:29:27.340059+0000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T23:29:28.520729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":27741},{"type":"v1","addr":"172.21.15.143:6801","nonce":27741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":27741},{"type":"v1","addr":"172.21.15.143:6803","nonce":27741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":27741},{"type":"v1","addr":"172.21.15.143:6807","nonce":27741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":27741},{"type":"v1","addr":"172.21.15.143:6805","nonce":27741}]},"public_addr":"172.21.15.143:6801/27741","cluster_addr":"172.21.15.143:6803/27741","heartbeat_back_addr":"172.21.15.143:6807/27741","heartbeat_front_addr":"172.21.15.143:6805/27741","state":["exists","up"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":30036},{"type":"v1","addr":"172.21.15.156:6817","nonce":30036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":30036},{"type":"v1","addr":"172.21.15.156:6819","nonce":30036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":30036},{"type":"v1","addr":"172.21.15.156:6823","nonce":30036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":30036},{"type":"v1","addr":"172.21.15.156:6821","nonce":30036}]},"public_addr":"172.21.15.156:6817/30036","cluster_addr":"172.21.15.156:6819/30036","heartbeat_back_addr":"172.21.15.156:6823/30036","heartbeat_front_addr":"172.21.15.156:6821/30036","state":["exists","up"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":30035},{"type":"v1","addr":"172.21.15.156:6809","nonce":30035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":30035},{"type":"v1","addr":"172.21.15.156:6811","nonce":30035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":30035},{"type":"v1","addr":"172.21.15.156:6815","nonce":30035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":30035},{"type":"v1","addr":"172.21.15.156:6813","nonce":30035}]},"public_addr":"172.21.15.156:6809/30035","cluster_addr":"172.21.15.156:6811/30035","heartbeat_back_addr":"172.21.15.156:6815/30035","heartbeat_front_addr":"172.21.15.156:6813/30035","state":["exists","up"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":30037},{"type":"v1","addr":"172.21.15.156:6801","nonce":30037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":30037},{"type":"v1","addr":"172.21.15.156:6803","nonce":30037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":30037},{"type":"v1","addr":"172.21.15.156:6807","nonce":30037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":30037},{"type":"v1","addr":"172.21.15.156:6805","nonce":30037}]},"public_addr":"172.21.15.156:6801/30037","cluster_addr":"172.21.15.156:6803/30037","heartbeat_back_addr":"172.21.15.156:6807/30037","heartbeat_front_addr":"172.21.15.156:6805/30037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.781977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.943955+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.806513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.624301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:29:33.558 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 10 mgrc shutdown 2023-09-29T23:29:33.558 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 -- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f742808e8e0 msgr2=0x7f7428090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:33.558 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f742808e8e0 0x7f7428090d90 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f744806d790 tx=0x7f7438001a20 comp rx=0 tx=0).stop 2023-09-29T23:29:33.558 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 -- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7448069d30 msgr2=0x7f744809cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:33.558 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7448069d30 0x7f744809cf70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f743000e8d0 tx=0x7f7430008040 comp rx=0 tx=0).stop 2023-09-29T23:29:33.558 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 -- 172.21.15.143:0/967964908 shutdown_connections 2023-09-29T23:29:33.558 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f742808e8e0 0x7f7428090d90 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.559 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f744813f980 0x7f744813e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.559 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f744806e910 0x7f744809d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.559 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 --2- 172.21.15.143:0/967964908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7448069d30 0x7f744809cf70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.559 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 -- 172.21.15.143:0/967964908 >> 172.21.15.143:0/967964908 conn(0x7f74480677f0 msgr2=0x7f744806c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:33.559 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 -- 172.21.15.143:0/967964908 shutdown_connections 2023-09-29T23:29:33.559 INFO:teuthology.misc.health.smithi143.stderr:2023-09-29T23:29:33.556+0000 7f744d699700 1 -- 172.21.15.143:0/967964908 wait complete. 2023-09-29T23:29:33.568 DEBUG:teuthology.misc:4 of 4 OSDs are up 2023-09-29T23:29:33.568 INFO:tasks.ceph:Creating RBD pool 2023-09-29T23:29:33.569 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd pool create rbd 8 2023-09-29T23:29:33.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 Processor -- start 2023-09-29T23:29:33.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 -- start start 2023-09-29T23:29:33.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c0a1710 0x7f595c0a1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595c09e850 con 0x7f595c09e480 2023-09-29T23:29:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595c0a1000 con 0x7f595c147880 2023-09-29T23:29:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c0a12c0 con 0x7f595c0a1710 2023-09-29T23:29:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5962b93700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c0a1710 0x7f595c0a1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5962392700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f595c09e480 0x7f595c0a0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54710/0 (socket says 172.21.15.143:54710) 2023-09-29T23:29:33.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5962392700 1 -- 172.21.15.143:0/3425917574 learned_addr learned my addr 172.21.15.143:0/3425917574 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:33.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5961390700 1 -- 172.21.15.143:0/3425917574 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3566684316 0 0) 0x7f595c09e850 con 0x7f595c09e480 2023-09-29T23:29:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5961390700 1 -- 172.21.15.143:0/3425917574 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5948003570 con 0x7f595c09e480 2023-09-29T23:29:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5962b93700 1 -- 172.21.15.143:0/3425917574 >> v1:172.21.15.156:6789/0 conn(0x7f595c147880 legacy=0x7f595c147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5962b93700 1 -- 172.21.15.143:0/3425917574 >> v1:172.21.15.143:6789/0 conn(0x7f595c09e480 legacy=0x7f595c0a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5962b93700 1 -- 172.21.15.143:0/3425917574 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c148700 con 0x7f595c0a1710 2023-09-29T23:29:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5962b93700 1 --2- 172.21.15.143:0/3425917574 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c0a1710 0x7f595c0a1bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5958009910 tx=0x7f59580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72eda173d250c5aa server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5961390700 1 -- 172.21.15.143:0/3425917574 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595800b680 con 0x7f595c0a1710 2023-09-29T23:29:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5961390700 1 -- 172.21.15.143:0/3425917574 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595800b7e0 con 0x7f595c0a1710 2023-09-29T23:29:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5961390700 1 -- 172.21.15.143:0/3425917574 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595800bb80 con 0x7f595c0a1710 2023-09-29T23:29:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 -- 172.21.15.143:0/3425917574 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c0a1710 msgr2=0x7f595c0a1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:33.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 --2- 172.21.15.143:0/3425917574 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c0a1710 0x7f595c0a1bb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5958009910 tx=0x7f59580091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 -- 172.21.15.143:0/3425917574 shutdown_connections 2023-09-29T23:29:33.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 --2- 172.21.15.143:0/3425917574 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c0a1710 0x7f595c0a1bb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 -- 172.21.15.143:0/3425917574 >> 172.21.15.143:0/3425917574 conn(0x7f595c069a60 msgr2=0x7f595c069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:33.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 -- 172.21.15.143:0/3425917574 shutdown_connections 2023-09-29T23:29:33.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.676+0000 7f5963394700 1 -- 172.21.15.143:0/3425917574 wait complete. 2023-09-29T23:29:33.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5963394700 1 Processor -- start 2023-09-29T23:29:33.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5963394700 1 -- start start 2023-09-29T23:29:33.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5963394700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c09e480 0x7f595c06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:33.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5963394700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f595c0a1710 0x7f595c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:33.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5963394700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c147880 0x7f595c134e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:33.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5963394700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c1576c0 con 0x7f595c0a1710 2023-09-29T23:29:33.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5963394700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c135750 con 0x7f595c09e480 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5963394700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595c138b80 con 0x7f595c147880 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5962b93700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c147880 0x7f595c134e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5962b93700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c147880 0x7f595c134e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60174/0 (socket says 172.21.15.143:60174) 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5962b93700 1 -- 172.21.15.143:0/2108435441 learned_addr learned my addr 172.21.15.143:0/2108435441 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5961b91700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f595c0a1710 0x7f595c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5962392700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c09e480 0x7f595c06d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5962b93700 1 -- 172.21.15.143:0/2108435441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c09e480 msgr2=0x7f595c06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5962b93700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c09e480 0x7f595c06d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5962b93700 1 -- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f595c0a1710 msgr2=0x7f595c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5962b93700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f595c0a1710 0x7f595c06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5962b93700 1 -- 172.21.15.143:0/2108435441 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5958009420 con 0x7f595c147880 2023-09-29T23:29:33.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5962b93700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c147880 0x7f595c134e40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f595c0684d0 tx=0x7f5958017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:33.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f594e7fc700 1 -- 172.21.15.143:0/2108435441 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595800bb60 con 0x7f595c147880 2023-09-29T23:29:33.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f594e7fc700 1 -- 172.21.15.143:0/2108435441 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5958027450 con 0x7f595c147880 2023-09-29T23:29:33.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5963394700 1 -- 172.21.15.143:0/2108435441 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c135380 con 0x7f595c147880 2023-09-29T23:29:33.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f594e7fc700 1 -- 172.21.15.143:0/2108435441 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958027830 con 0x7f595c147880 2023-09-29T23:29:33.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.680+0000 7f5963394700 1 -- 172.21.15.143:0/2108435441 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c06dd00 con 0x7f595c147880 2023-09-29T23:29:33.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.684+0000 7f5963394700 1 -- 172.21.15.143:0/2108435441 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595c06d880 con 0x7f595c147880 2023-09-29T23:29:33.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.684+0000 7f594e7fc700 1 -- 172.21.15.143:0/2108435441 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f595801a430 con 0x7f595c147880 2023-09-29T23:29:33.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.684+0000 7f594e7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:33.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.684+0000 7f594e7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:33.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.684+0000 7f594e7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:33.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.684+0000 7f594e7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:33.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.684+0000 7f594e7fc700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f594008e8b0 0x7f5940090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:33.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.684+0000 7f594e7fc700 1 -- 172.21.15.143:0/2108435441 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 3125+0+0 (secure 0 0 0) 0x7f59580b5eb0 con 0x7f595c147880 2023-09-29T23:29:33.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.684+0000 7f5962392700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f594008e8b0 0x7f5940090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:33.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.684+0000 7f5962392700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f594008e8b0 0x7f5940090d60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f59500058d0 tx=0x7f5950009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:33.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.688+0000 7f594e7fc700 1 -- 172.21.15.143:0/2108435441 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f595c06d880 con 0x7f595c147880 2023-09-29T23:29:33.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:33.848+0000 7f5963394700 1 -- 172.21.15.143:0/2108435441 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f595c06d880 con 0x7f595c147880 2023-09-29T23:29:34.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f594e7fc700 1 -- 172.21.15.143:0/2108435441 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v13) v1 ==== 109+0+0 (secure 0 0 0) 0x7f595c06d880 con 0x7f595c147880 2023-09-29T23:29:34.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 10 mgrc shutdown 2023-09-29T23:29:34.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 -- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f594008e8b0 msgr2=0x7f5940090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:34.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f594008e8b0 0x7f5940090d60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f59500058d0 tx=0x7f5950009040 comp rx=0 tx=0).stop 2023-09-29T23:29:34.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 -- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c147880 msgr2=0x7f595c134e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:34.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c147880 0x7f595c134e40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f595c0684d0 tx=0x7f5958017000 comp rx=0 tx=0).stop 2023-09-29T23:29:34.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 -- 172.21.15.143:0/2108435441 shutdown_connections 2023-09-29T23:29:34.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f594008e8b0 0x7f5940090d60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:34.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f595c147880 0x7f595c134e40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:34.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f595c0a1710 0x7f595c06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:34.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 --2- 172.21.15.143:0/2108435441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f595c09e480 0x7f595c06d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:34.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 -- 172.21.15.143:0/2108435441 >> 172.21.15.143:0/2108435441 conn(0x7f595c069a60 msgr2=0x7f595c0a1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:34.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 -- 172.21.15.143:0/2108435441 shutdown_connections 2023-09-29T23:29:34.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.088+0000 7f5963394700 1 -- 172.21.15.143:0/2108435441 wait complete. 2023-09-29T23:29:34.094 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'rbd' created 2023-09-29T23:29:34.106 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-09-29T23:29:34.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0359e700 1 Processor -- start 2023-09-29T23:29:34.210 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0359e700 1 -- start start 2023-09-29T23:29:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0359e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a00e0 0x7f0f040a2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0359e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f040a17d0 0x7f0f0413c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0359e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f040a0820 con 0x7f0f040a5470 2023-09-29T23:29:34.212 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0359e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f040a4be0 con 0x7f0f040a17d0 2023-09-29T23:29:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0359e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0413cec0 con 0x7f0f040a00e0 2023-09-29T23:29:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0259c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0f040a5470 0x7f0f040a44d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54712/0 (socket says 172.21.15.143:54712) 2023-09-29T23:29:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0259c700 1 -- 172.21.15.143:0/2729228137 learned_addr learned my addr 172.21.15.143:0/2729228137 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f01d9b700 1 --2- 172.21.15.143:0/2729228137 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a00e0 0x7f0f040a2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f02d9d700 1 --2- 172.21.15.143:0/2729228137 >> v2:172.21.15.156:3300/0 conn(0x7f0f040a17d0 0x7f0f0413c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0159a700 1 -- 172.21.15.143:0/2729228137 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084962983 0 0) 0x7f0f040a0820 con 0x7f0f040a5470 2023-09-29T23:29:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0159a700 1 -- 172.21.15.143:0/2729228137 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ef0003570 con 0x7f0f040a5470 2023-09-29T23:29:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f01d9b700 1 -- 172.21.15.143:0/2729228137 >> v2:172.21.15.156:3300/0 conn(0x7f0f040a17d0 msgr2=0x7f0f0413c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f01d9b700 1 --2- 172.21.15.143:0/2729228137 >> v2:172.21.15.156:3300/0 conn(0x7f0f040a17d0 0x7f0f0413c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f01d9b700 1 -- 172.21.15.143:0/2729228137 >> v1:172.21.15.143:6789/0 conn(0x7f0f040a5470 legacy=0x7f0f040a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:34.213 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f01d9b700 1 -- 172.21.15.143:0/2729228137 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0413d180 con 0x7f0f040a00e0 2023-09-29T23:29:34.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f01d9b700 1 --2- 172.21.15.143:0/2729228137 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a00e0 0x7f0f040a2970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0098e0 tx=0x7f0eec0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db7f860b34781377 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:34.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0159a700 1 -- 172.21.15.143:0/2729228137 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec011630 con 0x7f0f040a00e0 2023-09-29T23:29:34.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0159a700 1 -- 172.21.15.143:0/2729228137 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec011790 con 0x7f0f040a00e0 2023-09-29T23:29:34.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.208+0000 7f0f0159a700 1 -- 172.21.15.143:0/2729228137 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec011b30 con 0x7f0f040a00e0 2023-09-29T23:29:34.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 -- 172.21.15.143:0/2729228137 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a00e0 msgr2=0x7f0f040a2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:34.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 --2- 172.21.15.143:0/2729228137 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a00e0 0x7f0f040a2970 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0eec0098e0 tx=0x7f0eec0091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:34.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 -- 172.21.15.143:0/2729228137 shutdown_connections 2023-09-29T23:29:34.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 --2- 172.21.15.143:0/2729228137 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a00e0 0x7f0f040a2970 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:34.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 --2- 172.21.15.143:0/2729228137 >> v2:172.21.15.156:3300/0 conn(0x7f0f040a17d0 0x7f0f0413c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:34.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 -- 172.21.15.143:0/2729228137 >> 172.21.15.143:0/2729228137 conn(0x7f0f040b23c0 msgr2=0x7f0f040b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:34.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 -- 172.21.15.143:0/2729228137 shutdown_connections 2023-09-29T23:29:34.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 -- 172.21.15.143:0/2729228137 wait complete. 2023-09-29T23:29:34.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 Processor -- start 2023-09-29T23:29:34.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 -- start start 2023-09-29T23:29:34.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f040a00e0 0x7f0f0413f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:34.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f040a17d0 0x7f0f0406d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:34.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a5470 0x7f0f0406d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:34.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f04139700 con 0x7f0f040a17d0 2023-09-29T23:29:34.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0406db60 con 0x7f0f040a00e0 2023-09-29T23:29:34.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f04150b30 con 0x7f0f040a5470 2023-09-29T23:29:34.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f01d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f040a17d0 0x7f0f0406d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:34.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f02d9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a5470 0x7f0f0406d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:34.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f01d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f040a17d0 0x7f0f0406d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47676/0 (socket says 172.21.15.143:47676) 2023-09-29T23:29:34.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f02d9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a5470 0x7f0f0406d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60192/0 (socket says 172.21.15.143:60192) 2023-09-29T23:29:34.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f01d9b700 1 -- 172.21.15.143:0/575532831 learned_addr learned my addr 172.21.15.143:0/575532831 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:34.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0259c700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f040a00e0 0x7f0f0413f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:34.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f02d9d700 1 -- 172.21.15.143:0/575532831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f040a00e0 msgr2=0x7f0f0413f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:34.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f02d9d700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f040a00e0 0x7f0f0413f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:34.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f02d9d700 1 -- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f040a17d0 msgr2=0x7f0f0406d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:34.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f02d9d700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f040a17d0 0x7f0f0406d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:34.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f02d9d700 1 -- 172.21.15.143:0/575532831 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec009420 con 0x7f0f040a5470 2023-09-29T23:29:34.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f01d9b700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f040a17d0 0x7f0f0406d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:34.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0259c700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f040a00e0 0x7f0f0413f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:34.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f02d9d700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a5470 0x7f0f0406d620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00e980 tx=0x7f0efc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:34.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0efa7fc700 1 -- 172.21.15.143:0/575532831 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc00ee70 con 0x7f0f040a5470 2023-09-29T23:29:34.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0efa7fc700 1 -- 172.21.15.143:0/575532831 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0efc00c850 con 0x7f0f040a5470 2023-09-29T23:29:34.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 -- 172.21.15.143:0/575532831 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f04150870 con 0x7f0f040a5470 2023-09-29T23:29:34.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0efa7fc700 1 -- 172.21.15.143:0/575532831 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0efc00cc10 con 0x7f0f040a5470 2023-09-29T23:29:34.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.212+0000 7f0f0359e700 1 -- 172.21.15.143:0/575532831 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f04140220 con 0x7f0f040a5470 2023-09-29T23:29:34.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.216+0000 7f0f0359e700 1 -- 172.21.15.143:0/575532831 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f04139850 con 0x7f0f040a5470 2023-09-29T23:29:34.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.216+0000 7f0efa7fc700 1 -- 172.21.15.143:0/575532831 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f0efc013470 con 0x7f0f040a5470 2023-09-29T23:29:34.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.216+0000 7f0efa7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:34.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.220+0000 7f0efa7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:34.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.220+0000 7f0efa7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:34.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.220+0000 7f0efa7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:34.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.220+0000 7f0efa7fc700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ee408e8b0 0x7f0ee4090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:34.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.220+0000 7f0f0259c700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ee408e8b0 0x7f0ee4090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:34.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.220+0000 7f0efa7fc700 1 -- 172.21.15.143:0/575532831 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3467+0+0 (secure 0 0 0) 0x7f0efc0b4340 con 0x7f0f040a5470 2023-09-29T23:29:34.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.220+0000 7f0f0259c700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ee408e8b0 0x7f0ee4090d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005c50 tx=0x7f0ef4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:34.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.228+0000 7f0efa7fc700 1 -- 172.21.15.143:0/575532831 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0f04139850 con 0x7f0f040a5470 2023-09-29T23:29:34.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:34.384+0000 7f0f0359e700 1 -- 172.21.15.143:0/575532831 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0f04139850 con 0x7f0f040a5470 2023-09-29T23:29:35.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.088+0000 7f0efa7fc700 1 -- 172.21.15.143:0/575532831 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v14) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0f04139850 con 0x7f0f040a5470 2023-09-29T23:29:35.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 10 mgrc shutdown 2023-09-29T23:29:35.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 -- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ee408e8b0 msgr2=0x7f0ee4090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ee408e8b0 0x7f0ee4090d60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0ef4005c50 tx=0x7f0ef4009040 comp rx=0 tx=0).stop 2023-09-29T23:29:35.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 -- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a5470 msgr2=0x7f0f0406d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a5470 0x7f0f0406d620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0efc00e980 tx=0x7f0efc0091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 -- 172.21.15.143:0/575532831 shutdown_connections 2023-09-29T23:29:35.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ee408e8b0 0x7f0ee4090d60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0f040a5470 0x7f0f0406d620 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0f040a17d0 0x7f0f0406d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 --2- 172.21.15.143:0/575532831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f040a00e0 0x7f0f0413f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 -- 172.21.15.143:0/575532831 >> 172.21.15.143:0/575532831 conn(0x7f0f040b23c0 msgr2=0x7f0f040a3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:35.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 -- 172.21.15.143:0/575532831 shutdown_connections 2023-09-29T23:29:35.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.092+0000 7f0f0359e700 1 -- 172.21.15.143:0/575532831 wait complete. 2023-09-29T23:29:35.100 INFO:teuthology.orchestra.run.smithi143.stderr:enabled application 'rbd' on pool 'rbd' 2023-09-29T23:29:35.111 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-09-29T23:29:35.111 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-29T23:29:35.111 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-29T23:29:35.113 INFO:tasks.ceph.mds.a:Started 2023-09-29T23:29:35.113 INFO:tasks.ceph.mds.c:Restarting daemon 2023-09-29T23:29:35.113 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-09-29T23:29:35.115 INFO:tasks.ceph.mds.c:Started 2023-09-29T23:29:35.115 INFO:tasks.ceph.mds.b:Restarting daemon 2023-09-29T23:29:35.115 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-09-29T23:29:35.118 INFO:tasks.ceph.mds.b:Started 2023-09-29T23:29:35.118 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-09-29T23:29:35.118 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T23:29:35.177 INFO:tasks.ceph.mds.a.smithi143.stdout:starting mds.a at 2023-09-29T23:29:35.188 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2023-09-29T23:29:35.200 INFO:tasks.ceph.mds.c.smithi143.stdout:starting mds.c at 2023-09-29T23:29:35.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 Processor -- start 2023-09-29T23:29:35.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 -- start start 2023-09-29T23:29:35.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe0f806ee30 0x7fe0f80a1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe0f80a1c20 0x7fe0f813d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f813da50 con 0x7fe0f806f200 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f80a23c0 con 0x7fe0f80a1c20 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f80a2500 con 0x7fe0f806ee30 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f7fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe0f80a1c20 0x7fe0f813d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f7fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe0f80a1c20 0x7fe0f813d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47716/0 (socket says 172.21.15.143:47716) 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f6ffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe0f806f200 0x7fe0f813caf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54736/0 (socket says 172.21.15.143:54736) 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f7fff700 1 -- 172.21.15.143:0/4287670361 learned_addr learned my addr 172.21.15.143:0/4287670361 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f77fe700 1 --2- 172.21.15.143:0/4287670361 >> v2:172.21.15.156:3300/0 conn(0x7fe0f806ee30 0x7fe0f80a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f7fff700 1 -- 172.21.15.143:0/4287670361 >> v2:172.21.15.156:3300/0 conn(0x7fe0f806ee30 msgr2=0x7fe0f80a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f7fff700 1 --2- 172.21.15.143:0/4287670361 >> v2:172.21.15.156:3300/0 conn(0x7fe0f806ee30 0x7fe0f80a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f7fff700 1 -- 172.21.15.143:0/4287670361 >> v1:172.21.15.143:6789/0 conn(0x7fe0f806f200 legacy=0x7fe0f813caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f7fff700 1 -- 172.21.15.143:0/4287670361 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f80a2670 con 0x7fe0f80a1c20 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f67fc700 1 -- 172.21.15.143:0/4287670361 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210803767 0 0) 0x7fe0f813da50 con 0x7fe0f806f200 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f7fff700 1 --2- 172.21.15.143:0/4287670361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0f80a1c20 0x7fe0f813d510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8069f60 tx=0x7fe0ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d36aac60b5dffc server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f67fc700 1 -- 172.21.15.143:0/4287670361 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec00b860 con 0x7fe0f80a1c20 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f67fc700 1 -- 172.21.15.143:0/4287670361 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ec00b9c0 con 0x7fe0f80a1c20 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0f67fc700 1 -- 172.21.15.143:0/4287670361 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ec00bd60 con 0x7fe0f80a1c20 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/4287670361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0f80a1c20 msgr2=0x7fe0f813d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 --2- 172.21.15.143:0/4287670361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0f80a1c20 0x7fe0f813d510 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe0f8069f60 tx=0x7fe0ec009250 comp rx=0 tx=0).stop 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/4287670361 shutdown_connections 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 --2- 172.21.15.143:0/4287670361 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0f80a1c20 0x7fe0f813d510 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 --2- 172.21.15.143:0/4287670361 >> v2:172.21.15.156:3300/0 conn(0x7fe0f806ee30 0x7fe0f80a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/4287670361 >> 172.21.15.143:0/4287670361 conn(0x7fe0f806a6f0 msgr2=0x7fe0f80b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:35.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/4287670361 shutdown_connections 2023-09-29T23:29:35.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/4287670361 wait complete. 2023-09-29T23:29:35.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 Processor -- start 2023-09-29T23:29:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.216+0000 7fe0fdaf0700 1 -- start start 2023-09-29T23:29:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0fdaf0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0f806ee30 0x7fe0f8140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0fdaf0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0f806f200 0x7fe0f8097a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0fdaf0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f80a1c20 0x7fe0f8097fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0fdaf0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f8150f20 con 0x7fe0f806ee30 2023-09-29T23:29:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0fdaf0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81511c0 con 0x7fe0f80a1c20 2023-09-29T23:29:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0fdaf0700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f806e2a0 con 0x7fe0f806f200 2023-09-29T23:29:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f77fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0f806ee30 0x7fe0f8140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f77fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0f806ee30 0x7fe0f8140540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47718/0 (socket says 172.21.15.143:47718) 2023-09-29T23:29:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f6ffd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0f806f200 0x7fe0f8097a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f77fe700 1 -- 172.21.15.143:0/3317125333 learned_addr learned my addr 172.21.15.143:0/3317125333 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f7fff700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f80a1c20 0x7fe0f8097fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f6ffd700 1 -- 172.21.15.143:0/3317125333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f80a1c20 msgr2=0x7fe0f8097fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f6ffd700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f80a1c20 0x7fe0f8097fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f6ffd700 1 -- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0f806ee30 msgr2=0x7fe0f8140540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f6ffd700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0f806ee30 0x7fe0f8140540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f6ffd700 1 -- 172.21.15.143:0/3317125333 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec0094f0 con 0x7fe0f806f200 2023-09-29T23:29:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f77fe700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0f806ee30 0x7fe0f8140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0f6ffd700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0f806f200 0x7fe0f8097a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c5b0 tx=0x7fe0e8007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0df7fe700 1 -- 172.21.15.143:0/3317125333 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8005b40 con 0x7fe0f806f200 2023-09-29T23:29:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0df7fe700 1 -- 172.21.15.143:0/3317125333 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e8005ca0 con 0x7fe0f806f200 2023-09-29T23:29:35.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/3317125333 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f80972a0 con 0x7fe0f806f200 2023-09-29T23:29:35.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0df7fe700 1 -- 172.21.15.143:0/3317125333 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8007320 con 0x7fe0f806f200 2023-09-29T23:29:35.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/3317125333 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f80966d0 con 0x7fe0f806f200 2023-09-29T23:29:35.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.220+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/3317125333 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f805ecb0 con 0x7fe0f806f200 2023-09-29T23:29:35.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.224+0000 7fe0df7fe700 1 -- 172.21.15.143:0/3317125333 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fe0e801b3b0 con 0x7fe0f806f200 2023-09-29T23:29:35.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.224+0000 7fe0df7fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:35.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.224+0000 7fe0df7fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:35.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.224+0000 7fe0df7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:35.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.224+0000 7fe0df7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:35.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.224+0000 7fe0df7fe700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe0e408e8b0 0x7fe0e4090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.224+0000 7fe0f77fe700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe0e408e8b0 0x7fe0e4090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.224+0000 7fe0df7fe700 1 -- 172.21.15.143:0/3317125333 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3478+0+0 (secure 0 0 0) 0x7fe0e80b3900 con 0x7fe0f806f200 2023-09-29T23:29:35.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.224+0000 7fe0f77fe700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe0e408e8b0 0x7fe0e4090d60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0007af0 tx=0x7fe0f0009920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:35.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.232+0000 7fe0df7fe700 1 -- 172.21.15.143:0/3317125333 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe0e8084680 con 0x7fe0f806f200 2023-09-29T23:29:35.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.392+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/3317125333 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe0f80a28c0 con 0x7fe0f806f200 2023-09-29T23:29:35.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.392+0000 7fe0df7fe700 1 -- 172.21.15.143:0/3317125333 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+9003 (secure 0 0 0) 0x7fe0e8017030 con 0x7fe0f806f200 2023-09-29T23:29:35.397 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:35.397 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":14,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:29:35.080020+0000","last_up_change":"2023-09-29T23:29:27.340059+0000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T23:29:28.520729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T23:29:33.891889+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":27741},{"type":"v1","addr":"172.21.15.143:6801","nonce":27741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":27741},{"type":"v1","addr":"172.21.15.143:6803","nonce":27741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":27741},{"type":"v1","addr":"172.21.15.143:6807","nonce":27741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":27741},{"type":"v1","addr":"172.21.15.143:6805","nonce":27741}]},"public_addr":"172.21.15.143:6801/27741","cluster_addr":"172.21.15.143:6803/27741","heartbeat_back_addr":"172.21.15.143:6807/27741","heartbeat_front_addr":"172.21.15.143:6805/27741","state":["exists","up"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":30036},{"type":"v1","addr":"172.21.15.156:6817","nonce":30036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":30036},{"type":"v1","addr":"172.21.15.156:6819","nonce":30036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":30036},{"type":"v1","addr":"172.21.15.156:6823","nonce":30036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":30036},{"type":"v1","addr":"172.21.15.156:6821","nonce":30036}]},"public_addr":"172.21.15.156:6817/30036","cluster_addr":"172.21.15.156:6819/30036","heartbeat_back_addr":"172.21.15.156:6823/30036","heartbeat_front_addr":"172.21.15.156:6821/30036","state":["exists","up"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":30035},{"type":"v1","addr":"172.21.15.156:6809","nonce":30035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":30035},{"type":"v1","addr":"172.21.15.156:6811","nonce":30035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":30035},{"type":"v1","addr":"172.21.15.156:6815","nonce":30035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":30035},{"type":"v1","addr":"172.21.15.156:6813","nonce":30035}]},"public_addr":"172.21.15.156:6809/30035","cluster_addr":"172.21.15.156:6811/30035","heartbeat_back_addr":"172.21.15.156:6815/30035","heartbeat_front_addr":"172.21.15.156:6813/30035","state":["exists","up"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":30037},{"type":"v1","addr":"172.21.15.156:6801","nonce":30037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":30037},{"type":"v1","addr":"172.21.15.156:6803","nonce":30037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":30037},{"type":"v1","addr":"172.21.15.156:6807","nonce":30037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":30037},{"type":"v1","addr":"172.21.15.156:6805","nonce":30037}]},"public_addr":"172.21.15.156:6801/30037","cluster_addr":"172.21.15.156:6803/30037","heartbeat_back_addr":"172.21.15.156:6807/30037","heartbeat_front_addr":"172.21.15.156:6805/30037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.781977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.943955+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.806513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.624301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:29:35.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 10 mgrc shutdown 2023-09-29T23:29:35.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe0e408e8b0 msgr2=0x7fe0e4090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe0e408e8b0 0x7fe0e4090d60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0007af0 tx=0x7fe0f0009920 comp rx=0 tx=0).stop 2023-09-29T23:29:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0f806f200 msgr2=0x7fe0f8097a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0f806f200 0x7fe0f8097a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0e800c5b0 tx=0x7fe0e8007d50 comp rx=0 tx=0).stop 2023-09-29T23:29:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/3317125333 shutdown_connections 2023-09-29T23:29:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe0e408e8b0 0x7fe0e4090d60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0f80a1c20 0x7fe0f8097fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe0f806f200 0x7fe0f8097a80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 --2- 172.21.15.143:0/3317125333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe0f806ee30 0x7fe0f8140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/3317125333 >> 172.21.15.143:0/3317125333 conn(0x7fe0f806a6f0 msgr2=0x7fe0f80684b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:35.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/3317125333 shutdown_connections 2023-09-29T23:29:35.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.396+0000 7fe0fdaf0700 1 -- 172.21.15.143:0/3317125333 wait complete. 2023-09-29T23:29:35.417 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-29T23:29:28.520729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-29T23:29:33.891889+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-29T23:29:35.418 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-29T23:29:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c5312d700 1 Processor -- start 2023-09-29T23:29:35.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c5312d700 1 -- start start 2023-09-29T23:29:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c5312d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c4c066ec0 0x7f4c4c0a7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c5312d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c4c0a9850 0x7f4c4c0a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c5312d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c0a7940 con 0x7f4c4c0a88d0 2023-09-29T23:29:35.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c5312d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4c13dc00 con 0x7f4c4c066ec0 2023-09-29T23:29:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c5312d700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0aa2b0 con 0x7f4c4c0a9850 2023-09-29T23:29:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c50ec9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c4c066ec0 0x7f4c4c0a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c516ca700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c4c0a9850 0x7f4c4c0a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c50ec9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4c4c066ec0 0x7f4c4c0a7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47720/0 (socket says 172.21.15.143:47720) 2023-09-29T23:29:35.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c50ec9700 1 -- 172.21.15.143:0/2735279589 learned_addr learned my addr 172.21.15.143:0/2735279589 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c50ec9700 1 -- 172.21.15.143:0/2735279589 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c4c0a9850 msgr2=0x7f4c4c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c50ec9700 1 --2- 172.21.15.143:0/2735279589 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c4c0a9850 0x7f4c4c0a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c50ec9700 1 -- 172.21.15.143:0/2735279589 >> v1:172.21.15.156:6789/0 conn(0x7f4c4c0a88d0 legacy=0x7f4c4c141a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c50ec9700 1 -- 172.21.15.143:0/2735279589 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c067290 con 0x7f4c4c066ec0 2023-09-29T23:29:35.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c50ec9700 1 --2- 172.21.15.143:0/2735279589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c4c066ec0 0x7f4c4c0a7400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0099b0 tx=0x7f4c3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9712495c084c58 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c4b7fe700 1 -- 172.21.15.143:0/2735279589 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c009ea0 con 0x7f4c4c066ec0 2023-09-29T23:29:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c4b7fe700 1 -- 172.21.15.143:0/2735279589 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00b8f0 con 0x7f4c4c066ec0 2023-09-29T23:29:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.528+0000 7f4c4b7fe700 1 -- 172.21.15.143:0/2735279589 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00bc90 con 0x7f4c4c066ec0 2023-09-29T23:29:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 -- 172.21.15.143:0/2735279589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c4c066ec0 msgr2=0x7f4c4c0a7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 --2- 172.21.15.143:0/2735279589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c4c066ec0 0x7f4c4c0a7400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0099b0 tx=0x7f4c3c009250 comp rx=0 tx=0).stop 2023-09-29T23:29:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 -- 172.21.15.143:0/2735279589 shutdown_connections 2023-09-29T23:29:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 --2- 172.21.15.143:0/2735279589 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c4c0a9850 0x7f4c4c0a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 --2- 172.21.15.143:0/2735279589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c4c066ec0 0x7f4c4c0a7400 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 -- 172.21.15.143:0/2735279589 >> 172.21.15.143:0/2735279589 conn(0x7f4c4c068780 msgr2=0x7f4c4c06bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 -- 172.21.15.143:0/2735279589 shutdown_connections 2023-09-29T23:29:35.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 -- 172.21.15.143:0/2735279589 wait complete. 2023-09-29T23:29:35.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 Processor -- start 2023-09-29T23:29:35.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 -- start start 2023-09-29T23:29:35.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c4c066ec0 0x7f4c4c0b0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c4c0a88d0 0x7f4c4c0aea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c4c0a9850 0x7f4c4c140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c14dc50 con 0x7f4c4c0a9850 2023-09-29T23:29:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0afd40 con 0x7f4c4c066ec0 2023-09-29T23:29:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c5312d700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0af5d0 con 0x7f4c4c0a88d0 2023-09-29T23:29:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c4bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c4c0a88d0 0x7f4c4c0aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c516ca700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c4c0a9850 0x7f4c4c140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c4bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c4c0a88d0 0x7f4c4c0aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60248/0 (socket says 172.21.15.143:60248) 2023-09-29T23:29:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c4bfff700 1 -- 172.21.15.143:0/1551948485 learned_addr learned my addr 172.21.15.143:0/1551948485 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:35.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.532+0000 7f4c50ec9700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c4c066ec0 0x7f4c4c0b0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c516ca700 1 -- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c4c0a88d0 msgr2=0x7f4c4c0aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c516ca700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c4c0a88d0 0x7f4c4c0aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c516ca700 1 -- 172.21.15.143:0/1551948485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c4c066ec0 msgr2=0x7f4c4c0b0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c516ca700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c4c066ec0 0x7f4c4c0b0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c516ca700 1 -- 172.21.15.143:0/1551948485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0094f0 con 0x7f4c4c0a9850 2023-09-29T23:29:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c516ca700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c4c0a9850 0x7f4c4c140530 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400ecf0 tx=0x7f4c4400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c48ff9700 1 -- 172.21.15.143:0/1551948485 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400c8f0 con 0x7f4c4c0a9850 2023-09-29T23:29:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c48ff9700 1 -- 172.21.15.143:0/1551948485 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4400f040 con 0x7f4c4c0a9850 2023-09-29T23:29:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c5312d700 1 -- 172.21.15.143:0/1551948485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c0b06d0 con 0x7f4c4c0a9850 2023-09-29T23:29:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c48ff9700 1 -- 172.21.15.143:0/1551948485 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4400ccc0 con 0x7f4c4c0a9850 2023-09-29T23:29:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c5312d700 1 -- 172.21.15.143:0/1551948485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c140a70 con 0x7f4c4c0a9850 2023-09-29T23:29:35.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c5312d700 1 -- 172.21.15.143:0/1551948485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c140cc0 con 0x7f4c4c0a9850 2023-09-29T23:29:35.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c48ff9700 1 -- 172.21.15.143:0/1551948485 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f4c44014480 con 0x7f4c4c0a9850 2023-09-29T23:29:35.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.536+0000 7f4c48ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:35.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.540+0000 7f4c48ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:35.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.540+0000 7f4c48ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:35.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.540+0000 7f4c48ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:35.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.540+0000 7f4c48ff9700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c2c08e8b0 0x7f4c2c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.540+0000 7f4c50ec9700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c2c08e8b0 0x7f4c2c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.540+0000 7f4c48ff9700 1 -- 172.21.15.143:0/1551948485 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f4c440b4ea0 con 0x7f4c4c0a9850 2023-09-29T23:29:35.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.540+0000 7f4c50ec9700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c2c08e8b0 0x7f4c2c090d60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0099b0 tx=0x7f4c3c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:35.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.544+0000 7f4c48ff9700 1 -- 172.21.15.143:0/1551948485 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4c44085cb0 con 0x7f4c4c0a9850 2023-09-29T23:29:35.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.700+0000 7f4c5312d700 1 -- 172.21.15.143:0/1551948485 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c4c0afeb0 con 0x7f4c4c0a9850 2023-09-29T23:29:35.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.700+0000 7f4c48ff9700 1 -- 172.21.15.143:0/1551948485 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v14) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c44018020 con 0x7f4c4c0a9850 2023-09-29T23:29:35.703 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-09-29T23:29:35.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 10 mgrc shutdown 2023-09-29T23:29:35.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 -- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c2c08e8b0 msgr2=0x7f4c2c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c2c08e8b0 0x7f4c2c090d60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0099b0 tx=0x7f4c3c017000 comp rx=0 tx=0).stop 2023-09-29T23:29:35.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 -- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c4c0a9850 msgr2=0x7f4c4c140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c4c0a9850 0x7f4c4c140530 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400ecf0 tx=0x7f4c4400b040 comp rx=0 tx=0).stop 2023-09-29T23:29:35.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 -- 172.21.15.143:0/1551948485 shutdown_connections 2023-09-29T23:29:35.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c2c08e8b0 0x7f4c2c090d60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c4c0a9850 0x7f4c4c140530 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c4c0a88d0 0x7f4c4c0aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 --2- 172.21.15.143:0/1551948485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c4c066ec0 0x7f4c4c0b0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 -- 172.21.15.143:0/1551948485 >> 172.21.15.143:0/1551948485 conn(0x7f4c4c068780 msgr2=0x7f4c4c13d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:35.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 -- 172.21.15.143:0/1551948485 shutdown_connections 2023-09-29T23:29:35.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.704+0000 7f4c5312d700 1 -- 172.21.15.143:0/1551948485 wait complete. 2023-09-29T23:29:35.717 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-29T23:29:35.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.828+0000 7fb88359e700 1 Processor -- start 2023-09-29T23:29:35.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.828+0000 7fb88359e700 1 -- start start 2023-09-29T23:29:35.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.828+0000 7fb88359e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb884067ed0 0x7fb88413e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.828+0000 7fb88359e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8840682a0 con 0x7fb88409e650 2023-09-29T23:29:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.828+0000 7fb88359e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8840a6e90 con 0x7fb8840a0e90 2023-09-29T23:29:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.828+0000 7fb88359e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840a62a0 con 0x7fb884067ed0 2023-09-29T23:29:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.828+0000 7fb88259c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb884067ed0 0x7fb88413e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.828+0000 7fb881d9b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb88409e650 0x7fb88409f950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54750/0 (socket says 172.21.15.143:54750) 2023-09-29T23:29:35.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.828+0000 7fb881d9b700 1 -- 172.21.15.143:0/4279820014 learned_addr learned my addr 172.21.15.143:0/4279820014 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:35.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88159a700 1 -- 172.21.15.143:0/4279820014 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 927021445 0 0) 0x7fb8840682a0 con 0x7fb88409e650 2023-09-29T23:29:35.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88259c700 1 -- 172.21.15.143:0/4279820014 >> v1:172.21.15.156:6789/0 conn(0x7fb8840a0e90 legacy=0x7fb884148540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88259c700 1 -- 172.21.15.143:0/4279820014 >> v1:172.21.15.143:6789/0 conn(0x7fb88409e650 legacy=0x7fb88409f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88259c700 1 -- 172.21.15.143:0/4279820014 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8840a7150 con 0x7fb884067ed0 2023-09-29T23:29:35.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88259c700 1 --2- 172.21.15.143:0/4279820014 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb884067ed0 0x7fb88413e570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009910 tx=0x7fb86c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a9b7177bc03ad server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:35.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88159a700 1 -- 172.21.15.143:0/4279820014 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00b860 con 0x7fb884067ed0 2023-09-29T23:29:35.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88159a700 1 -- 172.21.15.143:0/4279820014 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c00b9c0 con 0x7fb884067ed0 2023-09-29T23:29:35.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88159a700 1 -- 172.21.15.143:0/4279820014 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c00bd60 con 0x7fb884067ed0 2023-09-29T23:29:35.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88359e700 1 -- 172.21.15.143:0/4279820014 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb884067ed0 msgr2=0x7fb88413e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88359e700 1 --2- 172.21.15.143:0/4279820014 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb884067ed0 0x7fb88413e570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009910 tx=0x7fb86c0091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88359e700 1 -- 172.21.15.143:0/4279820014 shutdown_connections 2023-09-29T23:29:35.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88359e700 1 --2- 172.21.15.143:0/4279820014 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb884067ed0 0x7fb88413e570 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88359e700 1 -- 172.21.15.143:0/4279820014 >> 172.21.15.143:0/4279820014 conn(0x7fb8840b45d0 msgr2=0x7fb8840b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:35.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88359e700 1 -- 172.21.15.143:0/4279820014 shutdown_connections 2023-09-29T23:29:35.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88359e700 1 -- 172.21.15.143:0/4279820014 wait complete. 2023-09-29T23:29:35.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.832+0000 7fb88359e700 1 Processor -- start 2023-09-29T23:29:35.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb88359e700 1 -- start start 2023-09-29T23:29:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb88359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb884067ed0 0x7fb8840a3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb88359e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb88409e650 0x7fb88413c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb88359e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8840a0e90 0x7fb884096910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb88359e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840a4660 con 0x7fb8840a0e90 2023-09-29T23:29:35.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb88359e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840a44f0 con 0x7fb884067ed0 2023-09-29T23:29:35.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb88359e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8840a1a30 con 0x7fb88409e650 2023-09-29T23:29:35.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb881d9b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb88409e650 0x7fb88413c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb882d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8840a0e90 0x7fb884096910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb881d9b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb88409e650 0x7fb88413c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60258/0 (socket says 172.21.15.143:60258) 2023-09-29T23:29:35.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb881d9b700 1 -- 172.21.15.143:0/1273029021 learned_addr learned my addr 172.21.15.143:0/1273029021 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:35.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb88259c700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb884067ed0 0x7fb8840a3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb881d9b700 1 -- 172.21.15.143:0/1273029021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb884067ed0 msgr2=0x7fb8840a3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb881d9b700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb884067ed0 0x7fb8840a3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb881d9b700 1 -- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8840a0e90 msgr2=0x7fb884096910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:35.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb881d9b700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8840a0e90 0x7fb884096910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:35.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb881d9b700 1 -- 172.21.15.143:0/1273029021 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86c009420 con 0x7fb88409e650 2023-09-29T23:29:35.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb881d9b700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb88409e650 0x7fb88413c0f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb87400c810 tx=0x7fb874007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:35.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb87a7fc700 1 -- 172.21.15.143:0/1273029021 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874005c00 con 0x7fb88409e650 2023-09-29T23:29:35.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb88359e700 1 -- 172.21.15.143:0/1273029021 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88413c720 con 0x7fb88409e650 2023-09-29T23:29:35.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb87a7fc700 1 -- 172.21.15.143:0/1273029021 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb874005d60 con 0x7fb88409e650 2023-09-29T23:29:35.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb88359e700 1 -- 172.21.15.143:0/1273029021 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8840a1570 con 0x7fb88409e650 2023-09-29T23:29:35.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.836+0000 7fb87a7fc700 1 -- 172.21.15.143:0/1273029021 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8740072e0 con 0x7fb88409e650 2023-09-29T23:29:35.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.840+0000 7fb88359e700 1 -- 172.21.15.143:0/1273029021 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb884096610 con 0x7fb88409e650 2023-09-29T23:29:35.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.844+0000 7fb87a7fc700 1 -- 172.21.15.143:0/1273029021 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fb87401b3b0 con 0x7fb88409e650 2023-09-29T23:29:35.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.844+0000 7fb87a7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:35.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.844+0000 7fb87a7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:35.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.844+0000 7fb87a7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:35.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.844+0000 7fb87a7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:35.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.844+0000 7fb87a7fc700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb86408e830 0x7fb864090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:35.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.844+0000 7fb88259c700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb86408e830 0x7fb864090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:35.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.844+0000 7fb87a7fc700 1 -- 172.21.15.143:0/1273029021 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3478+0+0 (secure 0 0 0) 0x7fb8740b3f10 con 0x7fb88409e650 2023-09-29T23:29:35.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.844+0000 7fb88259c700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb86408e830 0x7fb864090ce0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0093f0 tx=0x7fb86c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:35.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:35.848+0000 7fb87a7fc700 1 -- 172.21.15.143:0/1273029021 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb874084c90 con 0x7fb88409e650 2023-09-29T23:29:36.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 -- 172.21.15.143:0/1273029021 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb88413c720 con 0x7fb88409e650 2023-09-29T23:29:36.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb87a7fc700 1 -- 172.21.15.143:0/1273029021 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v14) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb88413c720 con 0x7fb88409e650 2023-09-29T23:29:36.014 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 8 2023-09-29T23:29:36.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 10 mgrc shutdown 2023-09-29T23:29:36.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 -- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb86408e830 msgr2=0x7fb864090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb86408e830 0x7fb864090ce0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0093f0 tx=0x7fb86c017000 comp rx=0 tx=0).stop 2023-09-29T23:29:36.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 -- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb88409e650 msgr2=0x7fb88413c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb88409e650 0x7fb88413c0f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb87400c810 tx=0x7fb874007d90 comp rx=0 tx=0).stop 2023-09-29T23:29:36.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 -- 172.21.15.143:0/1273029021 shutdown_connections 2023-09-29T23:29:36.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb86408e830 0x7fb864090ce0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb8840a0e90 0x7fb884096910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb88409e650 0x7fb88413c0f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 --2- 172.21.15.143:0/1273029021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb884067ed0 0x7fb8840a3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 -- 172.21.15.143:0/1273029021 >> 172.21.15.143:0/1273029021 conn(0x7fb8840b45d0 msgr2=0x7fb88409f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:36.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 -- 172.21.15.143:0/1273029021 shutdown_connections 2023-09-29T23:29:36.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.012+0000 7fb88359e700 1 -- 172.21.15.143:0/1273029021 wait complete. 2023-09-29T23:29:36.028 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-29T23:29:36.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 Processor -- start 2023-09-29T23:29:36.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 -- start start 2023-09-29T23:29:36.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc066c90 0x7f82bc069820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82bc067060 0x7f82bc13f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82bc069d60 con 0x7f82bc13fb80 2023-09-29T23:29:36.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82bc1443e0 con 0x7f82bc067060 2023-09-29T23:29:36.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc1434a0 con 0x7f82bc066c90 2023-09-29T23:29:36.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82ba7fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82bc067060 0x7f82bc13f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bb7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f82bc13fb80 0x7f82bc06ae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54764/0 (socket says 172.21.15.143:54764) 2023-09-29T23:29:36.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bb7fe700 1 -- 172.21.15.143:0/1602582336 learned_addr learned my addr 172.21.15.143:0/1602582336 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:36.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82baffd700 1 --2- 172.21.15.143:0/1602582336 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc066c90 0x7f82bc069820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82ba7fc700 1 -- 172.21.15.143:0/1602582336 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc066c90 msgr2=0x7f82bc069820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:36.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82ba7fc700 1 --2- 172.21.15.143:0/1602582336 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc066c90 0x7f82bc069820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82ba7fc700 1 -- 172.21.15.143:0/1602582336 >> v1:172.21.15.143:6789/0 conn(0x7f82bc13fb80 legacy=0x7f82bc06ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82b9ffb700 1 -- 172.21.15.143:0/1602582336 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209229906 0 0) 0x7f82b4001490 con 0x7f82bc13fb80 2023-09-29T23:29:36.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82ba7fc700 1 -- 172.21.15.143:0/1602582336 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82bc143760 con 0x7f82bc067060 2023-09-29T23:29:36.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82baffd700 1 --2- 172.21.15.143:0/1602582336 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc066c90 0x7f82bc069820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:36.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82ba7fc700 1 --2- 172.21.15.143:0/1602582336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82bc067060 0x7f82bc13f640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0098e0 tx=0x7f82ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614ee49f7fcd60aa server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:36.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82b9ffb700 1 -- 172.21.15.143:0/1602582336 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00b640 con 0x7f82bc067060 2023-09-29T23:29:36.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82b9ffb700 1 -- 172.21.15.143:0/1602582336 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac00b7a0 con 0x7f82bc067060 2023-09-29T23:29:36.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82b9ffb700 1 -- 172.21.15.143:0/1602582336 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00bb40 con 0x7f82bc067060 2023-09-29T23:29:36.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 -- 172.21.15.143:0/1602582336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82bc067060 msgr2=0x7f82bc13f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 --2- 172.21.15.143:0/1602582336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82bc067060 0x7f82bc13f640 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0098e0 tx=0x7f82ac0091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 -- 172.21.15.143:0/1602582336 shutdown_connections 2023-09-29T23:29:36.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 --2- 172.21.15.143:0/1602582336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82bc067060 0x7f82bc13f640 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.132+0000 7f82bbfff700 1 --2- 172.21.15.143:0/1602582336 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc066c90 0x7f82bc069820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82bbfff700 1 -- 172.21.15.143:0/1602582336 >> 172.21.15.143:0/1602582336 conn(0x7f82bc0b4820 msgr2=0x7f82bc0b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:36.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82bbfff700 1 -- 172.21.15.143:0/1602582336 shutdown_connections 2023-09-29T23:29:36.138 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82bbfff700 1 -- 172.21.15.143:0/1602582336 wait complete. 2023-09-29T23:29:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82bbfff700 1 Processor -- start 2023-09-29T23:29:36.139 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82bbfff700 1 -- start start 2023-09-29T23:29:36.141 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82bbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82bc066c90 0x7f82bc0b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82bbfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc067060 0x7f82bc0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82bbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82bc13fb80 0x7f82bc13bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82bbfff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc0b0c70 con 0x7f82bc13fb80 2023-09-29T23:29:36.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82bbfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc137bf0 con 0x7f82bc066c90 2023-09-29T23:29:36.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82bbfff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82bc13b6c0 con 0x7f82bc067060 2023-09-29T23:29:36.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.136+0000 7f82ba7fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc067060 0x7f82bc0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82ba7fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc067060 0x7f82bc0b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60278/0 (socket says 172.21.15.143:60278) 2023-09-29T23:29:36.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82ba7fc700 1 -- 172.21.15.143:0/103319191 learned_addr learned my addr 172.21.15.143:0/103319191 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:36.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82bb7fe700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82bc13fb80 0x7f82bc13bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82baffd700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82bc066c90 0x7f82bc0b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.142 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82ba7fc700 1 -- 172.21.15.143:0/103319191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82bc066c90 msgr2=0x7f82bc0b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82ba7fc700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82bc066c90 0x7f82bc0b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82ba7fc700 1 -- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82bc13fb80 msgr2=0x7f82bc13bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82ba7fc700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82bc13fb80 0x7f82bc13bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82ba7fc700 1 -- 172.21.15.143:0/103319191 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b0009420 con 0x7f82bc067060 2023-09-29T23:29:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82ba7fc700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc067060 0x7f82bc0b3830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f82bc06c8d0 tx=0x7f82ac017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:36.143 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82a2ffd700 1 -- 172.21.15.143:0/103319191 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac00bb20 con 0x7f82bc067060 2023-09-29T23:29:36.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82a2ffd700 1 -- 172.21.15.143:0/103319191 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac000c90 con 0x7f82bc067060 2023-09-29T23:29:36.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82bbfff700 1 -- 172.21.15.143:0/103319191 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ac009420 con 0x7f82bc067060 2023-09-29T23:29:36.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82a2ffd700 1 -- 172.21.15.143:0/103319191 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac011630 con 0x7f82bc067060 2023-09-29T23:29:36.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82bbfff700 1 -- 172.21.15.143:0/103319191 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82bc13c090 con 0x7f82bc067060 2023-09-29T23:29:36.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.140+0000 7f82bbfff700 1 -- 172.21.15.143:0/103319191 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82bc05ecb0 con 0x7f82bc067060 2023-09-29T23:29:36.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.144+0000 7f82a2ffd700 1 -- 172.21.15.143:0/103319191 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f82ac0119e0 con 0x7f82bc067060 2023-09-29T23:29:36.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.144+0000 7f82a2ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:36.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.144+0000 7f82a2ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:36.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.144+0000 7f82a2ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:36.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.144+0000 7f82a2ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:36.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.144+0000 7f82a2ffd700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f82a808e930 0x7f82a8090de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.144+0000 7f82baffd700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f82a808e930 0x7f82a8090de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.144+0000 7f82a2ffd700 1 -- 172.21.15.143:0/103319191 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f82ac0b59f0 con 0x7f82bc067060 2023-09-29T23:29:36.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.144+0000 7f82baffd700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f82a808e930 0x7f82a8090de0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009f40 tx=0x7f82b00091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:36.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.152+0000 7f82a2ffd700 1 -- 172.21.15.143:0/103319191 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f82ac086760 con 0x7f82bc067060 2023-09-29T23:29:36.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.348+0000 7f82bbfff700 1 -- 172.21.15.143:0/103319191 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82bc06dc70 con 0x7f82bc067060 2023-09-29T23:29:36.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82a2ffd700 1 -- 172.21.15.143:0/103319191 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2774 (secure 0 0 0) 0x7f82ac029af0 con 0x7f82bc067060 2023-09-29T23:29:36.354 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:36.354 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6825/2413469850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2413469850},{"type":"v1","addr":"172.21.15.156:6825","nonce":2413469850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6811/896631119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":896631119},{"type":"v1","addr":"172.21.15.143:6811","nonce":896631119}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4258,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6813/1935385308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1935385308},{"type":"v1","addr":"172.21.15.143:6813","nonce":1935385308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-09-29T23:29:36.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 10 mgrc shutdown 2023-09-29T23:29:36.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 -- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f82a808e930 msgr2=0x7f82a8090de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f82a808e930 0x7f82a8090de0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f82b0009f40 tx=0x7f82b00091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 -- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc067060 msgr2=0x7f82bc0b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc067060 0x7f82bc0b3830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f82bc06c8d0 tx=0x7f82ac017000 comp rx=0 tx=0).stop 2023-09-29T23:29:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 -- 172.21.15.143:0/103319191 shutdown_connections 2023-09-29T23:29:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f82a808e930 0x7f82a8090de0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82bc13fb80 0x7f82bc13bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82bc067060 0x7f82bc0b3830 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 --2- 172.21.15.143:0/103319191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82bc066c90 0x7f82bc0b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 -- 172.21.15.143:0/103319191 >> 172.21.15.143:0/103319191 conn(0x7f82bc0b4820 msgr2=0x7f82bc0b4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 -- 172.21.15.143:0/103319191 shutdown_connections 2023-09-29T23:29:36.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.352+0000 7f82bbfff700 1 -- 172.21.15.143:0/103319191 wait complete. 2023-09-29T23:29:36.358 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 3 2023-09-29T23:29:36.372 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T23:29:36.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.472+0000 7f827a77f700 1 Processor -- start 2023-09-29T23:29:36.477 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.472+0000 7f827a77f700 1 -- start start 2023-09-29T23:29:36.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82740675d0 0x7f82740a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f82740a9530 0x7f827414ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82740679a0 con 0x7f82740a9110 2023-09-29T23:29:36.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827406b4e0 con 0x7f82740675d0 2023-09-29T23:29:36.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f827406b7a0 con 0x7f82740a9530 2023-09-29T23:29:36.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f8273fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82740675d0 0x7f82740a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f82737fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f82740a9110 0x7f82741361e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54780/0 (socket says 172.21.15.143:54780) 2023-09-29T23:29:36.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f82737fe700 1 -- 172.21.15.143:0/3709770598 learned_addr learned my addr 172.21.15.143:0/3709770598 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:36.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f8278d1c700 1 --2- 172.21.15.143:0/3709770598 >> v2:172.21.15.156:3300/0 conn(0x7f82740a9530 0x7f827414ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f8273fff700 1 -- 172.21.15.143:0/3709770598 >> v2:172.21.15.156:3300/0 conn(0x7f82740a9530 msgr2=0x7f827414ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f8273fff700 1 --2- 172.21.15.143:0/3709770598 >> v2:172.21.15.156:3300/0 conn(0x7f82740a9530 0x7f827414ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f8272ffd700 1 -- 172.21.15.143:0/3709770598 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769441239 0 0) 0x7f82740679a0 con 0x7f82740a9110 2023-09-29T23:29:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f8273fff700 1 -- 172.21.15.143:0/3709770598 >> v1:172.21.15.143:6789/0 conn(0x7f82740a9110 legacy=0x7f82741361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f8273fff700 1 -- 172.21.15.143:0/3709770598 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82740aa5d0 con 0x7f82740675d0 2023-09-29T23:29:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f8273fff700 1 --2- 172.21.15.143:0/3709770598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82740675d0 0x7f82740a88c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f82740aa270 tx=0x7f82680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ef1d4e9b20b1c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:36.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f8272ffd700 1 -- 172.21.15.143:0/3709770598 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268011630 con 0x7f82740675d0 2023-09-29T23:29:36.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f8272ffd700 1 -- 172.21.15.143:0/3709770598 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8268011790 con 0x7f82740675d0 2023-09-29T23:29:36.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f8272ffd700 1 -- 172.21.15.143:0/3709770598 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268011b30 con 0x7f82740675d0 2023-09-29T23:29:36.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 -- 172.21.15.143:0/3709770598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82740675d0 msgr2=0x7f82740a88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 --2- 172.21.15.143:0/3709770598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82740675d0 0x7f82740a88c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f82740aa270 tx=0x7f82680091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 -- 172.21.15.143:0/3709770598 shutdown_connections 2023-09-29T23:29:36.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 --2- 172.21.15.143:0/3709770598 >> v2:172.21.15.156:3300/0 conn(0x7f82740a9530 0x7f827414ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 --2- 172.21.15.143:0/3709770598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82740675d0 0x7f82740a88c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 -- 172.21.15.143:0/3709770598 >> 172.21.15.143:0/3709770598 conn(0x7f82740b76e0 msgr2=0x7f8274066c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:36.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 -- 172.21.15.143:0/3709770598 shutdown_connections 2023-09-29T23:29:36.481 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.476+0000 7f827a77f700 1 -- 172.21.15.143:0/3709770598 wait complete. 2023-09-29T23:29:36.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f827a77f700 1 Processor -- start 2023-09-29T23:29:36.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f827a77f700 1 -- start start 2023-09-29T23:29:36.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f827a77f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82740675d0 0x7f82741441a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f827a77f700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82740a9110 0x7f8274130b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f827a77f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82740a9530 0x7f82740aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f827a77f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8274131070 con 0x7f82740a9530 2023-09-29T23:29:36.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f827a77f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8274131b50 con 0x7f82740675d0 2023-09-29T23:29:36.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f827a77f700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82740b0750 con 0x7f82740a9110 2023-09-29T23:29:36.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f8278d1c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82740a9530 0x7f82740aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f8278d1c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82740a9530 0x7f82740aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47772/0 (socket says 172.21.15.143:47772) 2023-09-29T23:29:36.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f8278d1c700 1 -- 172.21.15.143:0/3580312835 learned_addr learned my addr 172.21.15.143:0/3580312835 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:36.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f82737fe700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82740a9110 0x7f8274130b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f8273fff700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82740675d0 0x7f82741441a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f8278d1c700 1 -- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82740a9110 msgr2=0x7f8274130b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f8278d1c700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82740a9110 0x7f8274130b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f8278d1c700 1 -- 172.21.15.143:0/3580312835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82740675d0 msgr2=0x7f82741441a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f8278d1c700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82740675d0 0x7f82741441a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f8278d1c700 1 -- 172.21.15.143:0/3580312835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268009420 con 0x7f82740a9530 2023-09-29T23:29:36.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f8273fff700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82740675d0 0x7f82741441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:36.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f8278d1c700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82740a9530 0x7f82740aee30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f826c009e20 tx=0x7f826c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:36.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f825bfff700 1 -- 172.21.15.143:0/3580312835 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c00cb50 con 0x7f82740a9530 2023-09-29T23:29:36.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f825bfff700 1 -- 172.21.15.143:0/3580312835 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826c00f040 con 0x7f82740a9530 2023-09-29T23:29:36.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f827a77f700 1 -- 172.21.15.143:0/3580312835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82741312e0 con 0x7f82740a9530 2023-09-29T23:29:36.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f825bfff700 1 -- 172.21.15.143:0/3580312835 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c014480 con 0x7f82740a9530 2023-09-29T23:29:36.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.480+0000 7f827a77f700 1 -- 172.21.15.143:0/3580312835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82740acf20 con 0x7f82740a9530 2023-09-29T23:29:36.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.484+0000 7f827a77f700 1 -- 172.21.15.143:0/3580312835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82740b08e0 con 0x7f82740a9530 2023-09-29T23:29:36.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.484+0000 7f825bfff700 1 -- 172.21.15.143:0/3580312835 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f826c014830 con 0x7f82740a9530 2023-09-29T23:29:36.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.484+0000 7f825bfff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:36.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.484+0000 7f825bfff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:36.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.484+0000 7f825bfff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:36.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.484+0000 7f825bfff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:36.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.484+0000 7f825bfff700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f826008e8b0 0x7f8260090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.484+0000 7f8273fff700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f826008e8b0 0x7f8260090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.484+0000 7f825bfff700 1 -- 172.21.15.143:0/3580312835 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f826c0b4d10 con 0x7f82740a9530 2023-09-29T23:29:36.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.484+0000 7f8273fff700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f826008e8b0 0x7f8260090d60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f82680093f0 tx=0x7f8268017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:36.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.488+0000 7f825bfff700 1 -- 172.21.15.143:0/3580312835 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f826c085b20 con 0x7f82740a9530 2023-09-29T23:29:36.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.648+0000 7f827a77f700 1 -- 172.21.15.143:0/3580312835 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f827405ecb0 con 0x7f82740a9530 2023-09-29T23:29:36.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.648+0000 7f825bfff700 1 -- 172.21.15.143:0/3580312835 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+8990 (secure 0 0 0) 0x7f826c018020 con 0x7f82740a9530 2023-09-29T23:29:36.651 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:36.651 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":15,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:29:36.098539+0000","last_up_change":"2023-09-29T23:29:27.340059+0000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T23:29:28.520729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T23:29:33.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":27741},{"type":"v1","addr":"172.21.15.143:6801","nonce":27741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":27741},{"type":"v1","addr":"172.21.15.143:6803","nonce":27741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":27741},{"type":"v1","addr":"172.21.15.143:6807","nonce":27741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":27741},{"type":"v1","addr":"172.21.15.143:6805","nonce":27741}]},"public_addr":"172.21.15.143:6801/27741","cluster_addr":"172.21.15.143:6803/27741","heartbeat_back_addr":"172.21.15.143:6807/27741","heartbeat_front_addr":"172.21.15.143:6805/27741","state":["exists","up"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":30036},{"type":"v1","addr":"172.21.15.156:6817","nonce":30036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":30036},{"type":"v1","addr":"172.21.15.156:6819","nonce":30036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":30036},{"type":"v1","addr":"172.21.15.156:6823","nonce":30036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":30036},{"type":"v1","addr":"172.21.15.156:6821","nonce":30036}]},"public_addr":"172.21.15.156:6817/30036","cluster_addr":"172.21.15.156:6819/30036","heartbeat_back_addr":"172.21.15.156:6823/30036","heartbeat_front_addr":"172.21.15.156:6821/30036","state":["exists","up"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":30035},{"type":"v1","addr":"172.21.15.156:6809","nonce":30035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":30035},{"type":"v1","addr":"172.21.15.156:6811","nonce":30035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":30035},{"type":"v1","addr":"172.21.15.156:6815","nonce":30035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":30035},{"type":"v1","addr":"172.21.15.156:6813","nonce":30035}]},"public_addr":"172.21.15.156:6809/30035","cluster_addr":"172.21.15.156:6811/30035","heartbeat_back_addr":"172.21.15.156:6815/30035","heartbeat_front_addr":"172.21.15.156:6813/30035","state":["exists","up"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":13,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":30037},{"type":"v1","addr":"172.21.15.156:6801","nonce":30037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":30037},{"type":"v1","addr":"172.21.15.156:6803","nonce":30037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":30037},{"type":"v1","addr":"172.21.15.156:6807","nonce":30037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":30037},{"type":"v1","addr":"172.21.15.156:6805","nonce":30037}]},"public_addr":"172.21.15.156:6801/30037","cluster_addr":"172.21.15.156:6803/30037","heartbeat_back_addr":"172.21.15.156:6807/30037","heartbeat_front_addr":"172.21.15.156:6805/30037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.781977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.943955+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.806513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.624301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:29:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 10 mgrc shutdown 2023-09-29T23:29:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 -- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f826008e8b0 msgr2=0x7f8260090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.655 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f826008e8b0 0x7f8260090d60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f82680093f0 tx=0x7f8268017000 comp rx=0 tx=0).stop 2023-09-29T23:29:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 -- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82740a9530 msgr2=0x7f82740aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82740a9530 0x7f82740aee30 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f826c009e20 tx=0x7f826c00b040 comp rx=0 tx=0).stop 2023-09-29T23:29:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 -- 172.21.15.143:0/3580312835 shutdown_connections 2023-09-29T23:29:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f826008e8b0 0x7f8260090d60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82740a9530 0x7f82740aee30 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82740a9110 0x7f8274130b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 --2- 172.21.15.143:0/3580312835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82740675d0 0x7f82741441a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 -- 172.21.15.143:0/3580312835 >> 172.21.15.143:0/3580312835 conn(0x7f82740b76e0 msgr2=0x7f82741356c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 -- 172.21.15.143:0/3580312835 shutdown_connections 2023-09-29T23:29:36.656 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.652+0000 7f827a77f700 1 -- 172.21.15.143:0/3580312835 wait complete. 2023-09-29T23:29:36.670 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-29T23:29:28.520729+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-29T23:29:33.891889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-29T23:29:36.671 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-29T23:29:36.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2ded70f700 1 Processor -- start 2023-09-29T23:29:36.786 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2ded70f700 1 -- start start 2023-09-29T23:29:36.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2ded70f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2de806f060 0x7f2de80a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2ded70f700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de809eea0 0x7f2de809e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2ded70f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de809e560 con 0x7f2de81409d0 2023-09-29T23:29:36.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2ded70f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de809ea50 con 0x7f2de806f060 2023-09-29T23:29:36.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2ded70f700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de80a5810 con 0x7f2de809eea0 2023-09-29T23:29:36.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de77fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de809eea0 0x7f2de809e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2decf0e700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2de81409d0 0x7f2de80b1c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54796/0 (socket says 172.21.15.143:54796) 2023-09-29T23:29:36.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de7fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2de806f060 0x7f2de80a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2decf0e700 1 -- 172.21.15.143:0/3153446417 learned_addr learned my addr 172.21.15.143:0/3153446417 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:36.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de7fff700 1 -- 172.21.15.143:0/3153446417 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de809eea0 msgr2=0x7f2de809e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:36.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de7fff700 1 --2- 172.21.15.143:0/3153446417 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de809eea0 0x7f2de809e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de7fff700 1 -- 172.21.15.143:0/3153446417 >> v1:172.21.15.143:6789/0 conn(0x7f2de81409d0 legacy=0x7f2de80b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de6ffd700 1 -- 172.21.15.143:0/3153446417 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131913664 0 0) 0x7f2de809e560 con 0x7f2de81409d0 2023-09-29T23:29:36.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de7fff700 1 -- 172.21.15.143:0/3153446417 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de806f430 con 0x7f2de806f060 2023-09-29T23:29:36.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de77fe700 1 --2- 172.21.15.143:0/3153446417 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de809eea0 0x7f2de809e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:36.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de7fff700 1 --2- 172.21.15.143:0/3153446417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de806f060 0x7f2de80a5c70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009e60 tx=0x7f2dd8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5189aa09068716 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:36.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de6ffd700 1 -- 172.21.15.143:0/3153446417 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd800e070 con 0x7f2de806f060 2023-09-29T23:29:36.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de6ffd700 1 -- 172.21.15.143:0/3153446417 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd800bbf0 con 0x7f2de806f060 2023-09-29T23:29:36.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.784+0000 7f2de6ffd700 1 -- 172.21.15.143:0/3153446417 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8012430 con 0x7f2de806f060 2023-09-29T23:29:36.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.788+0000 7f2ded70f700 1 -- 172.21.15.143:0/3153446417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de806f060 msgr2=0x7f2de80a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.788+0000 7f2ded70f700 1 --2- 172.21.15.143:0/3153446417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de806f060 0x7f2de80a5c70 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009e60 tx=0x7f2dd8009780 comp rx=0 tx=0).stop 2023-09-29T23:29:36.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.788+0000 7f2ded70f700 1 -- 172.21.15.143:0/3153446417 shutdown_connections 2023-09-29T23:29:36.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.788+0000 7f2ded70f700 1 --2- 172.21.15.143:0/3153446417 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de809eea0 0x7f2de809e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.788+0000 7f2ded70f700 1 --2- 172.21.15.143:0/3153446417 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de806f060 0x7f2de80a5c70 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.788+0000 7f2ded70f700 1 -- 172.21.15.143:0/3153446417 >> 172.21.15.143:0/3153446417 conn(0x7f2de806a730 msgr2=0x7f2de80b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:36.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.788+0000 7f2ded70f700 1 -- 172.21.15.143:0/3153446417 shutdown_connections 2023-09-29T23:29:36.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.788+0000 7f2ded70f700 1 -- 172.21.15.143:0/3153446417 wait complete. 2023-09-29T23:29:36.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.788+0000 7f2ded70f700 1 Processor -- start 2023-09-29T23:29:36.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.788+0000 7f2ded70f700 1 -- start start 2023-09-29T23:29:36.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ded70f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de806f060 0x7f2de806be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.794 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ded70f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2de809eea0 0x7f2de81437b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ded70f700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de81409d0 0x7f2de813ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ded70f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de806c7c0 con 0x7f2de806f060 2023-09-29T23:29:36.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ded70f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de806ca80 con 0x7f2de809eea0 2023-09-29T23:29:36.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ded70f700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de809e3b0 con 0x7f2de81409d0 2023-09-29T23:29:36.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2decf0e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de81409d0 0x7f2de813ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2de7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de806f060 0x7f2de806be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2decf0e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de81409d0 0x7f2de813ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60312/0 (socket says 172.21.15.143:60312) 2023-09-29T23:29:36.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2decf0e700 1 -- 172.21.15.143:0/1305623412 learned_addr learned my addr 172.21.15.143:0/1305623412 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:36.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2de7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de806f060 0x7f2de806be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47778/0 (socket says 172.21.15.143:47778) 2023-09-29T23:29:36.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2de77fe700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2de809eea0 0x7f2de81437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2decf0e700 1 -- 172.21.15.143:0/1305623412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2de809eea0 msgr2=0x7f2de81437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2decf0e700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2de809eea0 0x7f2de81437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2decf0e700 1 -- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de806f060 msgr2=0x7f2de806be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2decf0e700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de806f060 0x7f2de806be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2decf0e700 1 -- 172.21.15.143:0/1305623412 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd0009420 con 0x7f2de81409d0 2023-09-29T23:29:36.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2decf0e700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de81409d0 0x7f2de813ba90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2de000c530 tx=0x7f2de0007f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:36.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ddffff700 1 -- 172.21.15.143:0/1305623412 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0005b30 con 0x7f2de81409d0 2023-09-29T23:29:36.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ddffff700 1 -- 172.21.15.143:0/1305623412 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de0005c90 con 0x7f2de81409d0 2023-09-29T23:29:36.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ddffff700 1 -- 172.21.15.143:0/1305623412 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0007320 con 0x7f2de81409d0 2023-09-29T23:29:36.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ded70f700 1 -- 172.21.15.143:0/1305623412 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd8009a20 con 0x7f2de81409d0 2023-09-29T23:29:36.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ded70f700 1 -- 172.21.15.143:0/1305623412 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de81421a0 con 0x7f2de81409d0 2023-09-29T23:29:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.792+0000 7f2ded70f700 1 -- 172.21.15.143:0/1305623412 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de8142330 con 0x7f2de81409d0 2023-09-29T23:29:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.796+0000 7f2ddffff700 1 -- 172.21.15.143:0/1305623412 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f2de001b3b0 con 0x7f2de81409d0 2023-09-29T23:29:36.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.796+0000 7f2ddffff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:36.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.796+0000 7f2ddffff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:36.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.796+0000 7f2ddffff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:36.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.796+0000 7f2ddffff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:36.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.796+0000 7f2ddffff700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2dc808e860 0x7f2dc8090d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:36.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.796+0000 7f2de7fff700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2dc808e860 0x7f2dc8090d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:36.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.796+0000 7f2ddffff700 1 -- 172.21.15.143:0/1305623412 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f2de00b38e0 con 0x7f2de81409d0 2023-09-29T23:29:36.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.796+0000 7f2de7fff700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2dc808e860 0x7f2dc8090d10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80097c0 tx=0x7f2dd8019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:36.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.804+0000 7f2ddffff700 1 -- 172.21.15.143:0/1305623412 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2de0084660 con 0x7f2de81409d0 2023-09-29T23:29:36.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.960+0000 7f2ded70f700 1 -- 172.21.15.143:0/1305623412 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2de805ecb0 con 0x7f2de81409d0 2023-09-29T23:29:36.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ddffff700 1 -- 172.21.15.143:0/1305623412 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v15) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2de0017030 con 0x7f2de81409d0 2023-09-29T23:29:36.965 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 1 2023-09-29T23:29:36.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 10 mgrc shutdown 2023-09-29T23:29:36.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 -- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2dc808e860 msgr2=0x7f2dc8090d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2dc808e860 0x7f2dc8090d10 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2dd80097c0 tx=0x7f2dd8019000 comp rx=0 tx=0).stop 2023-09-29T23:29:36.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 -- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de81409d0 msgr2=0x7f2de813ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:36.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de81409d0 0x7f2de813ba90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2de000c530 tx=0x7f2de0007f90 comp rx=0 tx=0).stop 2023-09-29T23:29:36.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 -- 172.21.15.143:0/1305623412 shutdown_connections 2023-09-29T23:29:36.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2dc808e860 0x7f2dc8090d10 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2de81409d0 0x7f2de813ba90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2de809eea0 0x7f2de81437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 --2- 172.21.15.143:0/1305623412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2de806f060 0x7f2de806be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:36.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 -- 172.21.15.143:0/1305623412 >> 172.21.15.143:0/1305623412 conn(0x7f2de806a730 msgr2=0x7f2de80b10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:36.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 -- 172.21.15.143:0/1305623412 shutdown_connections 2023-09-29T23:29:36.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:36.964+0000 7f2ded70f700 1 -- 172.21.15.143:0/1305623412 wait complete. 2023-09-29T23:29:36.979 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-29T23:29:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 Processor -- start 2023-09-29T23:29:37.094 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 -- start start 2023-09-29T23:29:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f02900b3650 0x7f0290148260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f029013a540 0x7f02900a4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900a4c50 0x7f0290066c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02900a5190 con 0x7f029013a540 2023-09-29T23:29:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02900a53c0 con 0x7f02900b3650 2023-09-29T23:29:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02900b3a20 con 0x7f02900a4c50 2023-09-29T23:29:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295207700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900a4c50 0x7f0290066c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0287fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f029013a540 0x7f02900a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295207700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900a4c50 0x7f0290066c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60320/0 (socket says 172.21.15.143:60320) 2023-09-29T23:29:37.095 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0287fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f029013a540 0x7f02900a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47790/0 (socket says 172.21.15.143:47790) 2023-09-29T23:29:37.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295207700 1 -- 172.21.15.143:0/816233483 learned_addr learned my addr 172.21.15.143:0/816233483 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:37.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0294a06700 1 --2- 172.21.15.143:0/816233483 >> v2:172.21.15.156:3300/0 conn(0x7f02900b3650 0x7f0290148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0287fff700 1 -- 172.21.15.143:0/816233483 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900a4c50 msgr2=0x7f0290066c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:37.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0287fff700 1 --2- 172.21.15.143:0/816233483 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900a4c50 0x7f0290066c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0287fff700 1 -- 172.21.15.143:0/816233483 >> v2:172.21.15.156:3300/0 conn(0x7f02900b3650 msgr2=0x7f0290148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:37.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0287fff700 1 --2- 172.21.15.143:0/816233483 >> v2:172.21.15.156:3300/0 conn(0x7f02900b3650 0x7f0290148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0287fff700 1 -- 172.21.15.143:0/816233483 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0290147be0 con 0x7f029013a540 2023-09-29T23:29:37.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0287fff700 1 --2- 172.21.15.143:0/816233483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f029013a540 0x7f02900a4710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0290147dd0 tx=0x7f02800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a3c947195ae2d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:37.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f02877fe700 1 -- 172.21.15.143:0/816233483 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280011630 con 0x7f029013a540 2023-09-29T23:29:37.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f02877fe700 1 -- 172.21.15.143:0/816233483 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0280011790 con 0x7f029013a540 2023-09-29T23:29:37.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f02877fe700 1 -- 172.21.15.143:0/816233483 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0280011b50 con 0x7f029013a540 2023-09-29T23:29:37.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 -- 172.21.15.143:0/816233483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f029013a540 msgr2=0x7f02900a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 --2- 172.21.15.143:0/816233483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f029013a540 0x7f02900a4710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0290147dd0 tx=0x7f02800091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 -- 172.21.15.143:0/816233483 shutdown_connections 2023-09-29T23:29:37.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 --2- 172.21.15.143:0/816233483 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900a4c50 0x7f0290066c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 --2- 172.21.15.143:0/816233483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f029013a540 0x7f02900a4710 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 --2- 172.21.15.143:0/816233483 >> v2:172.21.15.156:3300/0 conn(0x7f02900b3650 0x7f0290148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 -- 172.21.15.143:0/816233483 >> 172.21.15.143:0/816233483 conn(0x7f0290062c70 msgr2=0x7f0290063070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 -- 172.21.15.143:0/816233483 shutdown_connections 2023-09-29T23:29:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.092+0000 7f0295a08700 1 -- 172.21.15.143:0/816233483 wait complete. 2023-09-29T23:29:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0295a08700 1 Processor -- start 2023-09-29T23:29:37.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0295a08700 1 -- start start 2023-09-29T23:29:37.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0295a08700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02900a4c50 0x7f0290136580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0295a08700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900b3650 0x7f02900681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0295a08700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029013a540 0x7f029013be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0295a08700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0290136ac0 con 0x7f02900a4c50 2023-09-29T23:29:37.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0295a08700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029006dc80 con 0x7f029013a540 2023-09-29T23:29:37.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0295a08700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02900a0830 con 0x7f02900b3650 2023-09-29T23:29:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0287fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900b3650 0x7f02900681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0287fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900b3650 0x7f02900681b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60322/0 (socket says 172.21.15.143:60322) 2023-09-29T23:29:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0287fff700 1 -- 172.21.15.143:0/184082273 learned_addr learned my addr 172.21.15.143:0/184082273 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0294a06700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02900a4c50 0x7f0290136580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0295207700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029013a540 0x7f029013be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0287fff700 1 -- 172.21.15.143:0/184082273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029013a540 msgr2=0x7f029013be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0287fff700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029013a540 0x7f029013be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0287fff700 1 -- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02900a4c50 msgr2=0x7f0290136580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0287fff700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02900a4c50 0x7f0290136580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0287fff700 1 -- 172.21.15.143:0/184082273 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0280009420 con 0x7f02900b3650 2023-09-29T23:29:37.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0287fff700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900b3650 0x7f02900681b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0280013000 tx=0x7f0280016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:37.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0284ff9700 1 -- 172.21.15.143:0/184082273 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028001ad40 con 0x7f02900b3650 2023-09-29T23:29:37.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0284ff9700 1 -- 172.21.15.143:0/184082273 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028001f070 con 0x7f02900b3650 2023-09-29T23:29:37.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0295a08700 1 -- 172.21.15.143:0/184082273 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02900a0af0 con 0x7f02900b3650 2023-09-29T23:29:37.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0284ff9700 1 -- 172.21.15.143:0/184082273 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800266d0 con 0x7f02900b3650 2023-09-29T23:29:37.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.096+0000 7f0295a08700 1 -- 172.21.15.143:0/184082273 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0290145360 con 0x7f02900b3650 2023-09-29T23:29:37.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.100+0000 7f0295a08700 1 -- 172.21.15.143:0/184082273 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029005ecb0 con 0x7f02900b3650 2023-09-29T23:29:37.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.100+0000 7f0284ff9700 1 -- 172.21.15.143:0/184082273 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f028001b450 con 0x7f02900b3650 2023-09-29T23:29:37.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.100+0000 7f0284ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:37.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.100+0000 7f0284ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:37.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.100+0000 7f0284ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:37.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.100+0000 7f0284ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:37.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.100+0000 7f0284ff9700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f027c08e8e0 0x7f027c090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.100+0000 7f0294a06700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f027c08e8e0 0x7f027c090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.100+0000 7f0284ff9700 1 -- 172.21.15.143:0/184082273 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f0280021070 con 0x7f02900b3650 2023-09-29T23:29:37.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.100+0000 7f0294a06700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f027c08e8e0 0x7f027c090d90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f028c009db0 tx=0x7f028c0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:37.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.108+0000 7f0284ff9700 1 -- 172.21.15.143:0/184082273 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0280086ab0 con 0x7f02900b3650 2023-09-29T23:29:37.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.268+0000 7f0295a08700 1 -- 172.21.15.143:0/184082273 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f02900a3f30 con 0x7f02900b3650 2023-09-29T23:29:37.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.268+0000 7f0284ff9700 1 -- 172.21.15.143:0/184082273 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v15) v1 ==== 92+0+10 (secure 0 0 0) 0x7f028008a0d0 con 0x7f02900b3650 2023-09-29T23:29:37.271 INFO:teuthology.orchestra.run.smithi143.stdout:pg_num: 8 2023-09-29T23:29:37.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.268+0000 7f0295a08700 10 mgrc shutdown 2023-09-29T23:29:37.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.268+0000 7f0295a08700 1 -- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f027c08e8e0 msgr2=0x7f027c090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.272+0000 7f0295a08700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f027c08e8e0 0x7f027c090d90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f028c009db0 tx=0x7f028c0091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.272+0000 7f0295a08700 1 -- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900b3650 msgr2=0x7f02900681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.272+0000 7f0295a08700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900b3650 0x7f02900681b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0280013000 tx=0x7f0280016000 comp rx=0 tx=0).stop 2023-09-29T23:29:37.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.272+0000 7f0295a08700 1 -- 172.21.15.143:0/184082273 shutdown_connections 2023-09-29T23:29:37.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.272+0000 7f0295a08700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f027c08e8e0 0x7f027c090d90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.272+0000 7f0295a08700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f029013a540 0x7f029013be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.272+0000 7f0295a08700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02900b3650 0x7f02900681b0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.272+0000 7f0295a08700 1 --2- 172.21.15.143:0/184082273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f02900a4c50 0x7f0290136580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.272+0000 7f0295a08700 1 -- 172.21.15.143:0/184082273 >> 172.21.15.143:0/184082273 conn(0x7f0290062c70 msgr2=0x7f02900a5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:37.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.272+0000 7f0295a08700 1 -- 172.21.15.143:0/184082273 shutdown_connections 2023-09-29T23:29:37.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.272+0000 7f0295a08700 1 -- 172.21.15.143:0/184082273 wait complete. 2023-09-29T23:29:37.287 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-29T23:29:37.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2651c700 1 Processor -- start 2023-09-29T23:29:37.394 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2651c700 1 -- start start 2023-09-29T23:29:37.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2651c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1b200a01c0 0x7f1b200a0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2651c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a5270 0x7f1b201464e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2651c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b2009f480 con 0x7f1b2009e2c0 2023-09-29T23:29:37.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2651c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b200a2cc0 con 0x7f1b200a01c0 2023-09-29T23:29:37.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2651c700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b200a1080 con 0x7f1b200a5270 2023-09-29T23:29:37.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b24d19700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f1b2009e2c0 0x7f1b2009e690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54808/0 (socket says 172.21.15.143:54808) 2023-09-29T23:29:37.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b24d19700 1 -- 172.21.15.143:0/3346872854 learned_addr learned my addr 172.21.15.143:0/3346872854 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:37.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b25d1b700 1 --2- 172.21.15.143:0/3346872854 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a5270 0x7f1b201464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2551a700 1 --2- 172.21.15.143:0/3346872854 >> v2:172.21.15.143:3300/0 conn(0x7f1b200a01c0 0x7f1b200a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b1ffff700 1 -- 172.21.15.143:0/3346872854 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173713450 0 0) 0x7f1b2009f480 con 0x7f1b2009e2c0 2023-09-29T23:29:37.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b1ffff700 1 -- 172.21.15.143:0/3346872854 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b0c003570 con 0x7f1b2009e2c0 2023-09-29T23:29:37.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2551a700 1 -- 172.21.15.143:0/3346872854 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a5270 msgr2=0x7f1b201464e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:37.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2551a700 1 --2- 172.21.15.143:0/3346872854 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a5270 0x7f1b201464e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2551a700 1 -- 172.21.15.143:0/3346872854 >> v1:172.21.15.143:6789/0 conn(0x7f1b2009e2c0 legacy=0x7f1b2009e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2551a700 1 -- 172.21.15.143:0/3346872854 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2009ef30 con 0x7f1b200a01c0 2023-09-29T23:29:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b25d1b700 1 --2- 172.21.15.143:0/3346872854 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a5270 0x7f1b201464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b2551a700 1 --2- 172.21.15.143:0/3346872854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b200a01c0 0x7f1b200a0590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b100098e0 tx=0x7f1b100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b718baa094bc189 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b1ffff700 1 -- 172.21.15.143:0/3346872854 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10011630 con 0x7f1b200a01c0 2023-09-29T23:29:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.392+0000 7f1b1ffff700 1 -- 172.21.15.143:0/3346872854 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b10011790 con 0x7f1b200a01c0 2023-09-29T23:29:37.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b1ffff700 1 -- 172.21.15.143:0/3346872854 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10011b30 con 0x7f1b200a01c0 2023-09-29T23:29:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 -- 172.21.15.143:0/3346872854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b200a01c0 msgr2=0x7f1b200a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 --2- 172.21.15.143:0/3346872854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b200a01c0 0x7f1b200a0590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b100098e0 tx=0x7f1b100091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 -- 172.21.15.143:0/3346872854 shutdown_connections 2023-09-29T23:29:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 --2- 172.21.15.143:0/3346872854 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a5270 0x7f1b201464e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 --2- 172.21.15.143:0/3346872854 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b200a01c0 0x7f1b200a0590 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 -- 172.21.15.143:0/3346872854 >> 172.21.15.143:0/3346872854 conn(0x7f1b200b2ca0 msgr2=0x7f1b200b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 -- 172.21.15.143:0/3346872854 shutdown_connections 2023-09-29T23:29:37.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 -- 172.21.15.143:0/3346872854 wait complete. 2023-09-29T23:29:37.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 Processor -- start 2023-09-29T23:29:37.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 -- start start 2023-09-29T23:29:37.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b2009e2c0 0x7f1b2006cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a01c0 0x7f1b20142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b200a5270 0x7f1b20142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b201433f0 con 0x7f1b200a5270 2023-09-29T23:29:37.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b201438a0 con 0x7f1b2009e2c0 2023-09-29T23:29:37.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2651c700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b201435e0 con 0x7f1b200a01c0 2023-09-29T23:29:37.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b24d19700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a01c0 0x7f1b20142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b24d19700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a01c0 0x7f1b20142970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60328/0 (socket says 172.21.15.143:60328) 2023-09-29T23:29:37.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b24d19700 1 -- 172.21.15.143:0/1891550537 learned_addr learned my addr 172.21.15.143:0/1891550537 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:37.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b25d1b700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b200a5270 0x7f1b20142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.396+0000 7f1b2551a700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b2009e2c0 0x7f1b2006cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b24d19700 1 -- 172.21.15.143:0/1891550537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b2009e2c0 msgr2=0x7f1b2006cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b24d19700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b2009e2c0 0x7f1b2006cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b24d19700 1 -- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b200a5270 msgr2=0x7f1b20142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b24d19700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b200a5270 0x7f1b20142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b24d19700 1 -- 172.21.15.143:0/1891550537 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b18009420 con 0x7f1b200a01c0 2023-09-29T23:29:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b24d19700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a01c0 0x7f1b20142970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800cd30 tx=0x7f1b08001b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b1d7fa700 1 -- 172.21.15.143:0/1891550537 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0800ce90 con 0x7f1b200a01c0 2023-09-29T23:29:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b1d7fa700 1 -- 172.21.15.143:0/1891550537 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0800ab70 con 0x7f1b200a01c0 2023-09-29T23:29:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b2651c700 1 -- 172.21.15.143:0/1891550537 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b10009420 con 0x7f1b200a01c0 2023-09-29T23:29:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b1d7fa700 1 -- 172.21.15.143:0/1891550537 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b080070f0 con 0x7f1b200a01c0 2023-09-29T23:29:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b2651c700 1 -- 172.21.15.143:0/1891550537 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b20095a40 con 0x7f1b200a01c0 2023-09-29T23:29:37.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b2651c700 1 -- 172.21.15.143:0/1891550537 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2005ecb0 con 0x7f1b200a01c0 2023-09-29T23:29:37.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b1d7fa700 1 -- 172.21.15.143:0/1891550537 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f1b08007520 con 0x7f1b200a01c0 2023-09-29T23:29:37.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.400+0000 7f1b1d7fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:37.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.404+0000 7f1b1d7fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:37.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.404+0000 7f1b1d7fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:37.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.404+0000 7f1b1d7fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:37.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.404+0000 7f1b1d7fa700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1b0008e8b0 0x7f1b00090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.404+0000 7f1b1d7fa700 1 -- 172.21.15.143:0/1891550537 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f1b080b4900 con 0x7f1b200a01c0 2023-09-29T23:29:37.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.404+0000 7f1b2551a700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1b0008e8b0 0x7f1b00090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.404+0000 7f1b2551a700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1b0008e8b0 0x7f1b00090d60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b100091e0 tx=0x7f1b10017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:37.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.408+0000 7f1b1d7fa700 1 -- 172.21.15.143:0/1891550537 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1b08085680 con 0x7f1b200a01c0 2023-09-29T23:29:37.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.576+0000 7f1b2651c700 1 -- 172.21.15.143:0/1891550537 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1b2009bcd0 con 0x7f1b200a01c0 2023-09-29T23:29:37.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b1d7fa700 1 -- 172.21.15.143:0/1891550537 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v15) v1 ==== 84+0+126 (secure 0 0 0) 0x7f1b08088ca0 con 0x7f1b200a01c0 2023-09-29T23:29:37.581 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:37.581 INFO:teuthology.orchestra.run.smithi143.stdout:[ 2023-09-29T23:29:37.581 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-09-29T23:29:37.582 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 1, 2023-09-29T23:29:37.582 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": ".mgr" 2023-09-29T23:29:37.582 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2023-09-29T23:29:37.582 INFO:teuthology.orchestra.run.smithi143.stdout: { 2023-09-29T23:29:37.582 INFO:teuthology.orchestra.run.smithi143.stdout: "poolnum": 2, 2023-09-29T23:29:37.582 INFO:teuthology.orchestra.run.smithi143.stdout: "poolname": "rbd" 2023-09-29T23:29:37.583 INFO:teuthology.orchestra.run.smithi143.stdout: } 2023-09-29T23:29:37.583 INFO:teuthology.orchestra.run.smithi143.stdout:] 2023-09-29T23:29:37.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 10 mgrc shutdown 2023-09-29T23:29:37.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 -- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1b0008e8b0 msgr2=0x7f1b00090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1b0008e8b0 0x7f1b00090d60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1b100091e0 tx=0x7f1b10017000 comp rx=0 tx=0).stop 2023-09-29T23:29:37.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 -- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a01c0 msgr2=0x7f1b20142970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a01c0 0x7f1b20142970 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1b0800cd30 tx=0x7f1b08001b10 comp rx=0 tx=0).stop 2023-09-29T23:29:37.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 -- 172.21.15.143:0/1891550537 shutdown_connections 2023-09-29T23:29:37.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1b0008e8b0 0x7f1b00090d60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1b200a5270 0x7f1b20142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1b200a01c0 0x7f1b20142970 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 --2- 172.21.15.143:0/1891550537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b2009e2c0 0x7f1b2006cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 -- 172.21.15.143:0/1891550537 >> 172.21.15.143:0/1891550537 conn(0x7f1b200b2ca0 msgr2=0x7f1b200a3150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:37.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 -- 172.21.15.143:0/1891550537 shutdown_connections 2023-09-29T23:29:37.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.580+0000 7f1b2651c700 1 -- 172.21.15.143:0/1891550537 wait complete. 2023-09-29T23:29:37.595 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-29T23:29:37.595 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-29T23:29:37.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.700+0000 7f9c39844700 1 Processor -- start 2023-09-29T23:29:37.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39844700 1 -- start start 2023-09-29T23:29:37.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39844700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9c340a6010 0x7f9c3406c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39844700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9c34140350 0x7f9c341472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39844700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c340a30e0 con 0x7f9c3413ff50 2023-09-29T23:29:37.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39844700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3406c6f0 con 0x7f9c34140350 2023-09-29T23:29:37.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39844700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c340a63e0 con 0x7f9c340a6010 2023-09-29T23:29:37.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39043700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f9c34140350 0x7f9c341472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c2bfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f9c3413ff50 0x7f9c3413bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54822/0 (socket says 172.21.15.143:54822) 2023-09-29T23:29:37.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c2bfff700 1 -- 172.21.15.143:0/4154557613 learned_addr learned my addr 172.21.15.143:0/4154557613 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:37.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c38842700 1 --2- 172.21.15.143:0/4154557613 >> v2:172.21.15.156:3300/0 conn(0x7f9c340a6010 0x7f9c3406c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c2b7fe700 1 -- 172.21.15.143:0/4154557613 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3695654299 0 0) 0x7f9c340a30e0 con 0x7f9c3413ff50 2023-09-29T23:29:37.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39043700 1 -- 172.21.15.143:0/4154557613 >> v2:172.21.15.156:3300/0 conn(0x7f9c340a6010 msgr2=0x7f9c3406c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:37.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39043700 1 --2- 172.21.15.143:0/4154557613 >> v2:172.21.15.156:3300/0 conn(0x7f9c340a6010 0x7f9c3406c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39043700 1 -- 172.21.15.143:0/4154557613 >> v1:172.21.15.143:6789/0 conn(0x7f9c3413ff50 legacy=0x7f9c3413bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39043700 1 -- 172.21.15.143:0/4154557613 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3406cb50 con 0x7f9c34140350 2023-09-29T23:29:37.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c39043700 1 --2- 172.21.15.143:0/4154557613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c34140350 0x7f9c341472f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009ee0 tx=0x7f9c2c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a3d7f824e41153 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:37.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c2b7fe700 1 -- 172.21.15.143:0/4154557613 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c00de00 con 0x7f9c34140350 2023-09-29T23:29:37.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c2b7fe700 1 -- 172.21.15.143:0/4154557613 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c011460 con 0x7f9c34140350 2023-09-29T23:29:37.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.704+0000 7f9c2b7fe700 1 -- 172.21.15.143:0/4154557613 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c011820 con 0x7f9c34140350 2023-09-29T23:29:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.708+0000 7f9c39844700 1 -- 172.21.15.143:0/4154557613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c34140350 msgr2=0x7f9c341472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.708+0000 7f9c39844700 1 --2- 172.21.15.143:0/4154557613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c34140350 0x7f9c341472f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c009ee0 tx=0x7f9c2c009780 comp rx=0 tx=0).stop 2023-09-29T23:29:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.708+0000 7f9c39844700 1 -- 172.21.15.143:0/4154557613 shutdown_connections 2023-09-29T23:29:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.708+0000 7f9c39844700 1 --2- 172.21.15.143:0/4154557613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c34140350 0x7f9c341472f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.708+0000 7f9c39844700 1 --2- 172.21.15.143:0/4154557613 >> v2:172.21.15.156:3300/0 conn(0x7f9c340a6010 0x7f9c3406c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.708+0000 7f9c39844700 1 -- 172.21.15.143:0/4154557613 >> 172.21.15.143:0/4154557613 conn(0x7f9c3406abe0 msgr2=0x7f9c3406afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:37.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.708+0000 7f9c39844700 1 -- 172.21.15.143:0/4154557613 shutdown_connections 2023-09-29T23:29:37.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.708+0000 7f9c39844700 1 -- 172.21.15.143:0/4154557613 wait complete. 2023-09-29T23:29:37.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.708+0000 7f9c39844700 1 Processor -- start 2023-09-29T23:29:37.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.708+0000 7f9c39844700 1 -- start start 2023-09-29T23:29:37.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c39844700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c340a6010 0x7f9c340a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c39844700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c3413ff50 0x7f9c3409b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c39844700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34140350 0x7f9c34141750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c39844700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3409bbf0 con 0x7f9c340a6010 2023-09-29T23:29:37.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c39844700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c341412c0 con 0x7f9c34140350 2023-09-29T23:29:37.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c39844700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3414eed0 con 0x7f9c3413ff50 2023-09-29T23:29:37.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c38842700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c340a6010 0x7f9c340a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c2bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c3413ff50 0x7f9c3409b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c2bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c3413ff50 0x7f9c3409b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60340/0 (socket says 172.21.15.143:60340) 2023-09-29T23:29:37.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c2bfff700 1 -- 172.21.15.143:0/1415208933 learned_addr learned my addr 172.21.15.143:0/1415208933 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:37.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c38842700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c340a6010 0x7f9c340a4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47836/0 (socket says 172.21.15.143:47836) 2023-09-29T23:29:37.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c39043700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34140350 0x7f9c34141750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c2bfff700 1 -- 172.21.15.143:0/1415208933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34140350 msgr2=0x7f9c34141750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c2bfff700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34140350 0x7f9c34141750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c2bfff700 1 -- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c340a6010 msgr2=0x7f9c340a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:37.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c2bfff700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c340a6010 0x7f9c340a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:37.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c2bfff700 1 -- 172.21.15.143:0/1415208933 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c009a20 con 0x7f9c3413ff50 2023-09-29T23:29:37.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c39043700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34140350 0x7f9c34141750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:37.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c2bfff700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c3413ff50 0x7f9c3409b6b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400c7e0 tx=0x7f9c24007df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:37.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c28ff9700 1 -- 172.21.15.143:0/1415208933 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24005b30 con 0x7f9c3413ff50 2023-09-29T23:29:37.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c28ff9700 1 -- 172.21.15.143:0/1415208933 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c24005c90 con 0x7f9c3413ff50 2023-09-29T23:29:37.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c39844700 1 -- 172.21.15.143:0/1415208933 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3414f1f0 con 0x7f9c3413ff50 2023-09-29T23:29:37.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c28ff9700 1 -- 172.21.15.143:0/1415208933 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c24007300 con 0x7f9c3413ff50 2023-09-29T23:29:37.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c39844700 1 -- 172.21.15.143:0/1415208933 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c34097bd0 con 0x7f9c3413ff50 2023-09-29T23:29:37.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.712+0000 7f9c39844700 1 -- 172.21.15.143:0/1415208933 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3405ecb0 con 0x7f9c3413ff50 2023-09-29T23:29:37.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.716+0000 7f9c28ff9700 1 -- 172.21.15.143:0/1415208933 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f9c240076b0 con 0x7f9c3413ff50 2023-09-29T23:29:37.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.716+0000 7f9c28ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:37.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.716+0000 7f9c28ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:37.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.716+0000 7f9c28ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:37.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.716+0000 7f9c28ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:37.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.716+0000 7f9c28ff9700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9c2008e830 0x7f9c20090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:37.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.716+0000 7f9c28ff9700 1 -- 172.21.15.143:0/1415208933 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3478+0+0 (secure 0 0 0) 0x7f9c240b3af0 con 0x7f9c3413ff50 2023-09-29T23:29:37.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.716+0000 7f9c38842700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9c2008e830 0x7f9c20090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:37.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.716+0000 7f9c38842700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9c2008e830 0x7f9c20090ce0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009db0 tx=0x7f9c300091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:37.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.724+0000 7f9c28ff9700 1 -- 172.21.15.143:0/1415208933 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9c24084870 con 0x7f9c3413ff50 2023-09-29T23:29:37.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:37.884+0000 7f9c39844700 1 -- 172.21.15.143:0/1415208933 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9c341409b0 con 0x7f9c3413ff50 2023-09-29T23:29:38.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.236+0000 7f9c28ff9700 1 -- 172.21.15.143:0/1415208933 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v16) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9c24017030 con 0x7f9c3413ff50 2023-09-29T23:29:38.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.236+0000 7f9c39844700 10 mgrc shutdown 2023-09-29T23:29:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 -- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9c2008e830 msgr2=0x7f9c20090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9c2008e830 0x7f9c20090ce0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c30009db0 tx=0x7f9c300091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 -- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c3413ff50 msgr2=0x7f9c3409b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c3413ff50 0x7f9c3409b6b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9c2400c7e0 tx=0x7f9c24007df0 comp rx=0 tx=0).stop 2023-09-29T23:29:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 -- 172.21.15.143:0/1415208933 shutdown_connections 2023-09-29T23:29:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9c2008e830 0x7f9c20090ce0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c34140350 0x7f9c34141750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9c3413ff50 0x7f9c3409b6b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 --2- 172.21.15.143:0/1415208933 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c340a6010 0x7f9c340a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:38.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 -- 172.21.15.143:0/1415208933 >> 172.21.15.143:0/1415208933 conn(0x7f9c3406abe0 msgr2=0x7f9c3406d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 -- 172.21.15.143:0/1415208933 shutdown_connections 2023-09-29T23:29:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.240+0000 7f9c39844700 1 -- 172.21.15.143:0/1415208933 wait complete. 2023-09-29T23:29:38.243 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_metadata' created 2023-09-29T23:29:38.256 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-29T23:29:38.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76ffff700 1 Processor -- start 2023-09-29T23:29:38.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76ffff700 1 -- start start 2023-09-29T23:29:38.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa770149ab0 0x7fa77014be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:38.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76ffff700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7700a9150 con 0x7fa7700a55f0 2023-09-29T23:29:38.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76ffff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7700682a0 con 0x7fa770067ed0 2023-09-29T23:29:38.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76ffff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa770068b60 con 0x7fa770149ab0 2023-09-29T23:29:38.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76e7fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa7700a55f0 0x7fa77013f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54828/0 (socket says 172.21.15.143:54828) 2023-09-29T23:29:38.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76e7fc700 1 -- 172.21.15.143:0/2177994427 learned_addr learned my addr 172.21.15.143:0/2177994427 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:38.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76f7fe700 1 --2- 172.21.15.143:0/2177994427 >> v2:172.21.15.156:3300/0 conn(0x7fa770149ab0 0x7fa77014be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76dffb700 1 -- 172.21.15.143:0/2177994427 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68319501 0 0) 0x7fa7700a9150 con 0x7fa7700a55f0 2023-09-29T23:29:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76dffb700 1 -- 172.21.15.143:0/2177994427 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa758003570 con 0x7fa7700a55f0 2023-09-29T23:29:38.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76f7fe700 1 -- 172.21.15.143:0/2177994427 >> v1:172.21.15.156:6789/0 conn(0x7fa770067ed0 legacy=0x7fa7700a8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76f7fe700 1 -- 172.21.15.143:0/2177994427 >> v1:172.21.15.143:6789/0 conn(0x7fa7700a55f0 legacy=0x7fa77013f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76dffb700 1 -- 172.21.15.143:0/2177994427 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 42675954 0 0) 0x7fa758003570 con 0x7fa7700a55f0 2023-09-29T23:29:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.372+0000 7fa76f7fe700 1 -- 172.21.15.143:0/2177994427 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700a6870 con 0x7fa770149ab0 2023-09-29T23:29:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76f7fe700 1 --2- 172.21.15.143:0/2177994427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770149ab0 0x7fa77014be90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa768009e90 tx=0x7fa768009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41a52dea803f8f38 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76dffb700 1 -- 172.21.15.143:0/2177994427 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800e070 con 0x7fa770149ab0 2023-09-29T23:29:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76dffb700 1 -- 172.21.15.143:0/2177994427 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76800bc40 con 0x7fa770149ab0 2023-09-29T23:29:38.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76dffb700 1 -- 172.21.15.143:0/2177994427 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768012430 con 0x7fa770149ab0 2023-09-29T23:29:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76ffff700 1 -- 172.21.15.143:0/2177994427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770149ab0 msgr2=0x7fa77014be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76ffff700 1 --2- 172.21.15.143:0/2177994427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770149ab0 0x7fa77014be90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa768009e90 tx=0x7fa768009780 comp rx=0 tx=0).stop 2023-09-29T23:29:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76ffff700 1 -- 172.21.15.143:0/2177994427 shutdown_connections 2023-09-29T23:29:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76ffff700 1 --2- 172.21.15.143:0/2177994427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770149ab0 0x7fa77014be90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76ffff700 1 -- 172.21.15.143:0/2177994427 >> 172.21.15.143:0/2177994427 conn(0x7fa7700b6850 msgr2=0x7fa7700b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76ffff700 1 -- 172.21.15.143:0/2177994427 shutdown_connections 2023-09-29T23:29:38.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76ffff700 1 -- 172.21.15.143:0/2177994427 wait complete. 2023-09-29T23:29:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76ffff700 1 Processor -- start 2023-09-29T23:29:38.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.376+0000 7fa76ffff700 1 -- start start 2023-09-29T23:29:38.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770067ed0 0x7fa77013c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:38.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7700a55f0 0x7fa77013a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:38.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa770149ab0 0x7fa77013aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:38.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76ffff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa770139040 con 0x7fa7700a55f0 2023-09-29T23:29:38.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76ffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77013c740 con 0x7fa770067ed0 2023-09-29T23:29:38.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76ffff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77013b1c0 con 0x7fa770149ab0 2023-09-29T23:29:38.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7700a55f0 0x7fa77013a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:38.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7700a55f0 0x7fa77013a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47840/0 (socket says 172.21.15.143:47840) 2023-09-29T23:29:38.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76e7fc700 1 -- 172.21.15.143:0/4093868324 learned_addr learned my addr 172.21.15.143:0/4093868324 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:38.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76f7fe700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa770149ab0 0x7fa77013aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:38.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76effd700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770067ed0 0x7fa77013c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:38.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76e7fc700 1 -- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa770149ab0 msgr2=0x7fa77013aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:38.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76e7fc700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa770149ab0 0x7fa77013aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:38.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76e7fc700 1 -- 172.21.15.143:0/4093868324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770067ed0 msgr2=0x7fa77013c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:38.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76e7fc700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770067ed0 0x7fa77013c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:38.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76e7fc700 1 -- 172.21.15.143:0/4093868324 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768009a20 con 0x7fa7700a55f0 2023-09-29T23:29:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76e7fc700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7700a55f0 0x7fa77013a570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa76000c590 tx=0x7fa760007b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa756ffd700 1 -- 172.21.15.143:0/4093868324 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760007360 con 0x7fa7700a55f0 2023-09-29T23:29:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa756ffd700 1 -- 172.21.15.143:0/4093868324 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7600074e0 con 0x7fa7700a55f0 2023-09-29T23:29:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76ffff700 1 -- 172.21.15.143:0/4093868324 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77006ab50 con 0x7fa7700a55f0 2023-09-29T23:29:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa756ffd700 1 -- 172.21.15.143:0/4093868324 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760004830 con 0x7fa7700a55f0 2023-09-29T23:29:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76ffff700 1 -- 172.21.15.143:0/4093868324 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa770134e90 con 0x7fa7700a55f0 2023-09-29T23:29:38.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.380+0000 7fa76ffff700 1 -- 172.21.15.143:0/4093868324 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77005ecb0 con 0x7fa7700a55f0 2023-09-29T23:29:38.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.384+0000 7fa756ffd700 1 -- 172.21.15.143:0/4093868324 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fa760004ba0 con 0x7fa7700a55f0 2023-09-29T23:29:38.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.384+0000 7fa756ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:38.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.384+0000 7fa756ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:38.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.384+0000 7fa756ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:38.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.384+0000 7fa756ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:38.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.384+0000 7fa756ffd700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa75c08e8b0 0x7fa75c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:38.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.384+0000 7fa76effd700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa75c08e8b0 0x7fa75c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:38.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.384+0000 7fa756ffd700 1 -- 172.21.15.143:0/4093868324 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3848+0+0 (secure 0 0 0) 0x7fa7600b4da0 con 0x7fa7700a55f0 2023-09-29T23:29:38.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.384+0000 7fa76effd700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa75c08e8b0 0x7fa75c090d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa764003a80 tx=0x7fa764007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:38.392 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.388+0000 7fa756ffd700 1 -- 172.21.15.143:0/4093868324 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa760085910 con 0x7fa7700a55f0 2023-09-29T23:29:38.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:38.552+0000 7fa76ffff700 1 -- 172.21.15.143:0/4093868324 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa77004b5c0 con 0x7fa7700a55f0 2023-09-29T23:29:39.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.236+0000 7fa756ffd700 1 -- 172.21.15.143:0/4093868324 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v17) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa760088f30 con 0x7fa7700a55f0 2023-09-29T23:29:39.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 10 mgrc shutdown 2023-09-29T23:29:39.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 -- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa75c08e8b0 msgr2=0x7fa75c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa75c08e8b0 0x7fa75c090d60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa764003a80 tx=0x7fa764007ec0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 -- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7700a55f0 msgr2=0x7fa77013a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7700a55f0 0x7fa77013a570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa76000c590 tx=0x7fa760007b10 comp rx=0 tx=0).stop 2023-09-29T23:29:39.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 -- 172.21.15.143:0/4093868324 shutdown_connections 2023-09-29T23:29:39.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa75c08e8b0 0x7fa75c090d60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa770149ab0 0x7fa77013aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa7700a55f0 0x7fa77013a570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 --2- 172.21.15.143:0/4093868324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770067ed0 0x7fa77013c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 -- 172.21.15.143:0/4093868324 >> 172.21.15.143:0/4093868324 conn(0x7fa7700b6850 msgr2=0x7fa7700b5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:39.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 -- 172.21.15.143:0/4093868324 shutdown_connections 2023-09-29T23:29:39.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.240+0000 7fa76ffff700 1 -- 172.21.15.143:0/4093868324 wait complete. 2023-09-29T23:29:39.246 INFO:teuthology.orchestra.run.smithi143.stderr:pool 'cephfs_data' created 2023-09-29T23:29:39.259 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-29T23:29:39.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.360+0000 7f202e7f1700 1 Processor -- start 2023-09-29T23:29:39.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.360+0000 7f202e7f1700 1 -- start start 2023-09-29T23:29:39.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f20280630e0 0x7f20280b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 0x7f20280a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20280a55f0 con 0x7f20280634b0 2023-09-29T23:29:39.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20280a5890 con 0x7f20280630e0 2023-09-29T23:29:39.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20280a52d0 con 0x7f20280a4980 2023-09-29T23:29:39.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202cd8e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 0x7f20280a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202cd8e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 0x7f20280a4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60354/0 (socket says 172.21.15.143:60354) 2023-09-29T23:29:39.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202cd8e700 1 -- 172.21.15.143:0/1016650389 learned_addr learned my addr 172.21.15.143:0/1016650389 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:39.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f2027fff700 1 --2- 172.21.15.143:0/1016650389 >> v2:172.21.15.156:3300/0 conn(0x7f20280630e0 0x7f20280b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202cd8e700 1 -- 172.21.15.143:0/1016650389 >> v2:172.21.15.156:3300/0 conn(0x7f20280630e0 msgr2=0x7f20280b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:39.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202cd8e700 1 --2- 172.21.15.143:0/1016650389 >> v2:172.21.15.156:3300/0 conn(0x7f20280630e0 0x7f20280b0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202cd8e700 1 -- 172.21.15.143:0/1016650389 >> v1:172.21.15.156:6789/0 conn(0x7f20280634b0 legacy=0x7f20280a7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202cd8e700 1 -- 172.21.15.143:0/1016650389 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028134880 con 0x7f20280a4980 2023-09-29T23:29:39.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202cd8e700 1 --2- 172.21.15.143:0/1016650389 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 0x7f20280a4d90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2020009f20 tx=0x7f2020009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22e334f459c6d19d server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:39.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f2026ffd700 1 -- 172.21.15.143:0/1016650389 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202000e070 con 0x7f20280a4980 2023-09-29T23:29:39.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f2026ffd700 1 -- 172.21.15.143:0/1016650389 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f202000b9d0 con 0x7f20280a4980 2023-09-29T23:29:39.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f2026ffd700 1 -- 172.21.15.143:0/1016650389 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20200092e0 con 0x7f20280a4980 2023-09-29T23:29:39.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 -- 172.21.15.143:0/1016650389 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 msgr2=0x7f20280a4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 --2- 172.21.15.143:0/1016650389 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 0x7f20280a4d90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2020009f20 tx=0x7f2020009840 comp rx=0 tx=0).stop 2023-09-29T23:29:39.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 -- 172.21.15.143:0/1016650389 shutdown_connections 2023-09-29T23:29:39.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 --2- 172.21.15.143:0/1016650389 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 0x7f20280a4d90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 --2- 172.21.15.143:0/1016650389 >> v2:172.21.15.156:3300/0 conn(0x7f20280630e0 0x7f20280b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 -- 172.21.15.143:0/1016650389 >> 172.21.15.143:0/1016650389 conn(0x7f202806ad20 msgr2=0x7f202806b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:39.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 -- 172.21.15.143:0/1016650389 shutdown_connections 2023-09-29T23:29:39.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.364+0000 7f202e7f1700 1 -- 172.21.15.143:0/1016650389 wait complete. 2023-09-29T23:29:39.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f202e7f1700 1 Processor -- start 2023-09-29T23:29:39.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f202e7f1700 1 -- start start 2023-09-29T23:29:39.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f202e7f1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20280630e0 0x7f20280ae9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f202e7f1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20280634b0 0x7f20280af970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f202e7f1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 0x7f2028124d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f202e7f1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280af360 con 0x7f20280634b0 2023-09-29T23:29:39.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f202e7f1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20280aef00 con 0x7f20280630e0 2023-09-29T23:29:39.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f202e7f1700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20280b0340 con 0x7f20280a4980 2023-09-29T23:29:39.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f202cd8e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 0x7f2028124d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f2027fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20280630e0 0x7f20280ae9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f2027fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20280630e0 0x7f20280ae9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.143:43172/0 (socket says 172.21.15.143:43172) 2023-09-29T23:29:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f2027fff700 1 -- 172.21.15.143:0/491430639 learned_addr learned my addr 172.21.15.143:0/491430639 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f20277fe700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20280634b0 0x7f20280af970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f2027fff700 1 -- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 msgr2=0x7f2028124d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f2027fff700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 0x7f2028124d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f2027fff700 1 -- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20280634b0 msgr2=0x7f20280af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f2027fff700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20280634b0 0x7f20280af970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.368+0000 7f2027fff700 1 -- 172.21.15.143:0/491430639 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2020009ae0 con 0x7f20280630e0 2023-09-29T23:29:39.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.372+0000 7f2027fff700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20280630e0 0x7f20280ae9c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f201800ecb0 tx=0x7f20180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:39.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.372+0000 7f200ffff700 1 -- 172.21.15.143:0/491430639 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201800c850 con 0x7f20280630e0 2023-09-29T23:29:39.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.372+0000 7f200ffff700 1 -- 172.21.15.143:0/491430639 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201800c9b0 con 0x7f20280630e0 2023-09-29T23:29:39.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.372+0000 7f202e7f1700 1 -- 172.21.15.143:0/491430639 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202814dd40 con 0x7f20280630e0 2023-09-29T23:29:39.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.372+0000 7f200ffff700 1 -- 172.21.15.143:0/491430639 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f201800cd50 con 0x7f20280630e0 2023-09-29T23:29:39.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.372+0000 7f202e7f1700 1 -- 172.21.15.143:0/491430639 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20280b00a0 con 0x7f20280630e0 2023-09-29T23:29:39.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.372+0000 7f202e7f1700 1 -- 172.21.15.143:0/491430639 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20280af7c0 con 0x7f20280630e0 2023-09-29T23:29:39.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.376+0000 7f200ffff700 1 -- 172.21.15.143:0/491430639 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f2018013680 con 0x7f20280630e0 2023-09-29T23:29:39.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.376+0000 7f200ffff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:39.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.376+0000 7f200ffff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:39.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.376+0000 7f200ffff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:39.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.376+0000 7f200ffff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:39.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.376+0000 7f200ffff700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f201408e930 0x7f2014090de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.376+0000 7f20277fe700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f201408e930 0x7f2014090de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.376+0000 7f200ffff700 1 -- 172.21.15.143:0/491430639 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4230+0+0 (secure 0 0 0) 0x7f20180b5230 con 0x7f20280630e0 2023-09-29T23:29:39.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.376+0000 7f20277fe700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f201408e930 0x7f2014090de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2028068dd0 tx=0x7f201c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:39.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.384+0000 7f200ffff700 1 -- 172.21.15.143:0/491430639 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2018085c20 con 0x7f20280630e0 2023-09-29T23:29:39.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.580+0000 7f202e7f1700 1 -- 172.21.15.143:0/491430639 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f20280af5e0 con 0x7f20280630e0 2023-09-29T23:29:39.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.596+0000 7f200ffff700 1 -- 172.21.15.143:0/491430639 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f20180210c0 con 0x7f20280630e0 2023-09-29T23:29:39.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 10 mgrc shutdown 2023-09-29T23:29:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 -- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f201408e930 msgr2=0x7f2014090de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f201408e930 0x7f2014090de0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2028068dd0 tx=0x7f201c009040 comp rx=0 tx=0).stop 2023-09-29T23:29:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 -- 172.21.15.143:0/491430639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20280630e0 msgr2=0x7f20280ae9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20280630e0 0x7f20280ae9c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f201800ecb0 tx=0x7f20180091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 -- 172.21.15.143:0/491430639 shutdown_connections 2023-09-29T23:29:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f201408e930 0x7f2014090de0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20280a4980 0x7f2028124d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20280634b0 0x7f20280af970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 --2- 172.21.15.143:0/491430639 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20280630e0 0x7f20280ae9c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 -- 172.21.15.143:0/491430639 >> 172.21.15.143:0/491430639 conn(0x7f202806ad20 msgr2=0x7f20281324d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:39.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 -- 172.21.15.143:0/491430639 shutdown_connections 2023-09-29T23:29:39.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.600+0000 7f202e7f1700 1 -- 172.21.15.143:0/491430639 wait complete. 2023-09-29T23:29:39.604 INFO:teuthology.orchestra.run.smithi143.stderr:new fs with metadata pool 3 and data pool 4 2023-09-29T23:29:39.615 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T23:29:39.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.708+0000 7f277a46c700 1 Processor -- start 2023-09-29T23:29:39.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.708+0000 7f277a46c700 1 -- start start 2023-09-29T23:29:39.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f27740b3710 0x7f27741363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2774136920 0x7f277409da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27740b3ae0 con 0x7f277409dfb0 2023-09-29T23:29:39.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f277409e3a0 con 0x7f27740b3710 2023-09-29T23:29:39.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2774144700 con 0x7f2774136920 2023-09-29T23:29:39.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277946a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f27740b3710 0x7f27741363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f2779c6b700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f277409dfb0 0x7f2774143a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54840/0 (socket says 172.21.15.143:54840) 2023-09-29T23:29:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277946a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f27740b3710 0x7f27741363e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47866/0 (socket says 172.21.15.143:47866) 2023-09-29T23:29:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f2779c6b700 1 -- 172.21.15.143:0/3919369631 learned_addr learned my addr 172.21.15.143:0/3919369631 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f2778c69700 1 --2- 172.21.15.143:0/3919369631 >> v2:172.21.15.156:3300/0 conn(0x7f2774136920 0x7f277409da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277946a700 1 -- 172.21.15.143:0/3919369631 >> v2:172.21.15.156:3300/0 conn(0x7f2774136920 msgr2=0x7f277409da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277946a700 1 --2- 172.21.15.143:0/3919369631 >> v2:172.21.15.156:3300/0 conn(0x7f2774136920 0x7f277409da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277946a700 1 -- 172.21.15.143:0/3919369631 >> v1:172.21.15.143:6789/0 conn(0x7f277409dfb0 legacy=0x7f2774143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277946a700 1 -- 172.21.15.143:0/3919369631 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27740a5bc0 con 0x7f27740b3710 2023-09-29T23:29:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277946a700 1 --2- 172.21.15.143:0/3919369631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27740b3710 0x7f27741363e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f277406ba20 tx=0x7f27640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3785a24e6eafd271 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:39.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f276bfff700 1 -- 172.21.15.143:0/3919369631 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764011630 con 0x7f27740b3710 2023-09-29T23:29:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f276bfff700 1 -- 172.21.15.143:0/3919369631 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2764011790 con 0x7f27740b3710 2023-09-29T23:29:39.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f276bfff700 1 -- 172.21.15.143:0/3919369631 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764011b30 con 0x7f27740b3710 2023-09-29T23:29:39.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 -- 172.21.15.143:0/3919369631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27740b3710 msgr2=0x7f27741363e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 --2- 172.21.15.143:0/3919369631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27740b3710 0x7f27741363e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f277406ba20 tx=0x7f27640091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 -- 172.21.15.143:0/3919369631 shutdown_connections 2023-09-29T23:29:39.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 --2- 172.21.15.143:0/3919369631 >> v2:172.21.15.156:3300/0 conn(0x7f2774136920 0x7f277409da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 --2- 172.21.15.143:0/3919369631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27740b3710 0x7f27741363e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 -- 172.21.15.143:0/3919369631 >> 172.21.15.143:0/3919369631 conn(0x7f2774062ed0 msgr2=0x7f27740632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:39.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 -- 172.21.15.143:0/3919369631 shutdown_connections 2023-09-29T23:29:39.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 -- 172.21.15.143:0/3919369631 wait complete. 2023-09-29T23:29:39.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 Processor -- start 2023-09-29T23:29:39.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.712+0000 7f277a46c700 1 -- start start 2023-09-29T23:29:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f277a46c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f277409dfb0 0x7f277406c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f277a46c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27740b3710 0x7f277409b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f277a46c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2774136920 0x7f27740a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f277a46c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2774150280 con 0x7f277409dfb0 2023-09-29T23:29:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f277a46c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f277406bdb0 con 0x7f27740b3710 2023-09-29T23:29:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f277a46c700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27740a0ee0 con 0x7f2774136920 2023-09-29T23:29:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f2779c6b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2774136920 0x7f27740a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f2779c6b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2774136920 0x7f27740a04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60370/0 (socket says 172.21.15.143:60370) 2023-09-29T23:29:39.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f2779c6b700 1 -- 172.21.15.143:0/3315599300 learned_addr learned my addr 172.21.15.143:0/3315599300 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:39.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f277946a700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f277409dfb0 0x7f277406c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f2778c69700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27740b3710 0x7f277409b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f2779c6b700 1 -- 172.21.15.143:0/3315599300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27740b3710 msgr2=0x7f277409b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f2779c6b700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27740b3710 0x7f277409b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f2779c6b700 1 -- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f277409dfb0 msgr2=0x7f277406c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f2779c6b700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f277409dfb0 0x7f277406c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f2779c6b700 1 -- 172.21.15.143:0/3315599300 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2764009420 con 0x7f2774136920 2023-09-29T23:29:39.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f2778c69700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27740b3710 0x7f277409b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:39.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f2779c6b700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2774136920 0x7f27740a04f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f276c00c590 tx=0x7f276c007ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:39.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f27697fa700 1 -- 172.21.15.143:0/3315599300 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c005c00 con 0x7f2774136920 2023-09-29T23:29:39.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f27697fa700 1 -- 172.21.15.143:0/3315599300 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c005d80 con 0x7f2774136920 2023-09-29T23:29:39.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f277a46c700 1 -- 172.21.15.143:0/3315599300 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27740a1200 con 0x7f2774136920 2023-09-29T23:29:39.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f27697fa700 1 -- 172.21.15.143:0/3315599300 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c007320 con 0x7f2774136920 2023-09-29T23:29:39.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f277a46c700 1 -- 172.21.15.143:0/3315599300 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f277409c370 con 0x7f2774136920 2023-09-29T23:29:39.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.716+0000 7f277a46c700 1 -- 172.21.15.143:0/3315599300 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277406cbf0 con 0x7f2774136920 2023-09-29T23:29:39.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.720+0000 7f27697fa700 1 -- 172.21.15.143:0/3315599300 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f276c01b3b0 con 0x7f2774136920 2023-09-29T23:29:39.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.720+0000 7f27697fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:39.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.720+0000 7f27697fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:39.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.720+0000 7f27697fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:39.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.720+0000 7f27697fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:39.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.720+0000 7f27697fa700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f275808e8b0 0x7f2758090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:39.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.720+0000 7f27697fa700 1 -- 172.21.15.143:0/3315599300 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f276c0b3dc0 con 0x7f2774136920 2023-09-29T23:29:39.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.720+0000 7f277946a700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f275808e8b0 0x7f2758090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:39.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.720+0000 7f277946a700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f275808e8b0 0x7f2758090d60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27640093f0 tx=0x7f2764017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:39.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.724+0000 7f27697fa700 1 -- 172.21.15.143:0/3315599300 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f276c0847f0 con 0x7f2774136920 2023-09-29T23:29:39.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.884+0000 7f277a46c700 1 -- 172.21.15.143:0/3315599300 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f277405ecb0 con 0x7f2774136920 2023-09-29T23:29:39.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f27697fa700 1 -- 172.21.15.143:0/3315599300 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12219 (secure 0 0 0) 0x7f276c017070 con 0x7f2774136920 2023-09-29T23:29:39.890 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:39.891 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":18,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:29:39.586692+0000","last_up_change":"2023-09-29T23:29:27.340059+0000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T23:29:28.520729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T23:29:33.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-29T23:29:37.927418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-29T23:29:38.594861+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":27741},{"type":"v1","addr":"172.21.15.143:6801","nonce":27741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":27741},{"type":"v1","addr":"172.21.15.143:6803","nonce":27741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":27741},{"type":"v1","addr":"172.21.15.143:6807","nonce":27741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":27741},{"type":"v1","addr":"172.21.15.143:6805","nonce":27741}]},"public_addr":"172.21.15.143:6801/27741","cluster_addr":"172.21.15.143:6803/27741","heartbeat_back_addr":"172.21.15.143:6807/27741","heartbeat_front_addr":"172.21.15.143:6805/27741","state":["exists","up"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":30036},{"type":"v1","addr":"172.21.15.156:6817","nonce":30036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":30036},{"type":"v1","addr":"172.21.15.156:6819","nonce":30036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":30036},{"type":"v1","addr":"172.21.15.156:6823","nonce":30036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":30036},{"type":"v1","addr":"172.21.15.156:6821","nonce":30036}]},"public_addr":"172.21.15.156:6817/30036","cluster_addr":"172.21.15.156:6819/30036","heartbeat_back_addr":"172.21.15.156:6823/30036","heartbeat_front_addr":"172.21.15.156:6821/30036","state":["exists","up"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":30035},{"type":"v1","addr":"172.21.15.156:6809","nonce":30035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":30035},{"type":"v1","addr":"172.21.15.156:6811","nonce":30035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":30035},{"type":"v1","addr":"172.21.15.156:6815","nonce":30035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":30035},{"type":"v1","addr":"172.21.15.156:6813","nonce":30035}]},"public_addr":"172.21.15.156:6809/30035","cluster_addr":"172.21.15.156:6811/30035","heartbeat_back_addr":"172.21.15.156:6815/30035","heartbeat_front_addr":"172.21.15.156:6813/30035","state":["exists","up"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":30037},{"type":"v1","addr":"172.21.15.156:6801","nonce":30037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":30037},{"type":"v1","addr":"172.21.15.156:6803","nonce":30037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":30037},{"type":"v1","addr":"172.21.15.156:6807","nonce":30037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":30037},{"type":"v1","addr":"172.21.15.156:6805","nonce":30037}]},"public_addr":"172.21.15.156:6801/30037","cluster_addr":"172.21.15.156:6803/30037","heartbeat_back_addr":"172.21.15.156:6807/30037","heartbeat_front_addr":"172.21.15.156:6805/30037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.781977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.943955+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.806513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.624301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:29:39.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 10 mgrc shutdown 2023-09-29T23:29:39.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 -- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f275808e8b0 msgr2=0x7f2758090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f275808e8b0 0x7f2758090d60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f27640093f0 tx=0x7f2764017000 comp rx=0 tx=0).stop 2023-09-29T23:29:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 -- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2774136920 msgr2=0x7f27740a04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2774136920 0x7f27740a04f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f276c00c590 tx=0x7f276c007ef0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 -- 172.21.15.143:0/3315599300 shutdown_connections 2023-09-29T23:29:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f275808e8b0 0x7f2758090d60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2774136920 0x7f27740a04f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27740b3710 0x7f277409b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 --2- 172.21.15.143:0/3315599300 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f277409dfb0 0x7f277406c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 -- 172.21.15.143:0/3315599300 >> 172.21.15.143:0/3315599300 conn(0x7f2774062ed0 msgr2=0x7f27740684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 -- 172.21.15.143:0/3315599300 shutdown_connections 2023-09-29T23:29:39.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:39.888+0000 7f277a46c700 1 -- 172.21.15.143:0/3315599300 wait complete. 2023-09-29T23:29:39.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T23:29:40.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302bfff700 1 Processor -- start 2023-09-29T23:29:40.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302bfff700 1 -- start start 2023-09-29T23:29:40.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f302c0a9c70 0x7f302c0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 0x7f302c1489d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302bfff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302c0a90a0 con 0x7f302c0a98a0 2023-09-29T23:29:40.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302bfff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302c0b4430 con 0x7f302c0a9c70 2023-09-29T23:29:40.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302bfff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0aa270 con 0x7f302c1465f0 2023-09-29T23:29:40.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302b7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 0x7f302c1489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302b7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 0x7f302c1489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60374/0 (socket says 172.21.15.143:60374) 2023-09-29T23:29:40.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302b7fe700 1 -- 172.21.15.143:0/1243793258 learned_addr learned my addr 172.21.15.143:0/1243793258 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:40.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302a7fc700 1 --2- 172.21.15.143:0/1243793258 >> v2:172.21.15.156:3300/0 conn(0x7f302c0a9c70 0x7f302c0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302b7fe700 1 -- 172.21.15.143:0/1243793258 >> v2:172.21.15.156:3300/0 conn(0x7f302c0a9c70 msgr2=0x7f302c0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:40.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302b7fe700 1 --2- 172.21.15.143:0/1243793258 >> v2:172.21.15.156:3300/0 conn(0x7f302c0a9c70 0x7f302c0b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302b7fe700 1 -- 172.21.15.143:0/1243793258 >> v1:172.21.15.156:6789/0 conn(0x7f302c0a98a0 legacy=0x7f302c13d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302b7fe700 1 -- 172.21.15.143:0/1243793258 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c13e030 con 0x7f302c1465f0 2023-09-29T23:29:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.012+0000 7f302b7fe700 1 --2- 172.21.15.143:0/1243793258 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 0x7f302c1489d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3024009f20 tx=0x7f3024009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4b31f5aa4d5ca53 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f3029ffb700 1 -- 172.21.15.143:0/1243793258 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302400e070 con 0x7f302c1465f0 2023-09-29T23:29:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f3029ffb700 1 -- 172.21.15.143:0/1243793258 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302400b9d0 con 0x7f302c1465f0 2023-09-29T23:29:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f3029ffb700 1 -- 172.21.15.143:0/1243793258 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240092e0 con 0x7f302c1465f0 2023-09-29T23:29:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f302bfff700 1 -- 172.21.15.143:0/1243793258 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 msgr2=0x7f302c1489d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f302bfff700 1 --2- 172.21.15.143:0/1243793258 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 0x7f302c1489d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3024009f20 tx=0x7f3024009840 comp rx=0 tx=0).stop 2023-09-29T23:29:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f302bfff700 1 -- 172.21.15.143:0/1243793258 shutdown_connections 2023-09-29T23:29:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f302bfff700 1 --2- 172.21.15.143:0/1243793258 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 0x7f302c1489d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f302bfff700 1 --2- 172.21.15.143:0/1243793258 >> v2:172.21.15.156:3300/0 conn(0x7f302c0a9c70 0x7f302c0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f302bfff700 1 -- 172.21.15.143:0/1243793258 >> 172.21.15.143:0/1243793258 conn(0x7f302c0b4730 msgr2=0x7f302c0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:40.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f302bfff700 1 -- 172.21.15.143:0/1243793258 shutdown_connections 2023-09-29T23:29:40.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f302bfff700 1 -- 172.21.15.143:0/1243793258 wait complete. 2023-09-29T23:29:40.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f302bfff700 1 Processor -- start 2023-09-29T23:29:40.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f302bfff700 1 -- start start 2023-09-29T23:29:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.016+0000 7f302bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f302c0a98a0 0x7f302c0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c0a9c70 0x7f302c14daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 0x7f302c14dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302bfff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0b02d0 con 0x7f302c0a9c70 2023-09-29T23:29:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302bfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c14e9a0 con 0x7f302c0a98a0 2023-09-29T23:29:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302bfff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0af660 con 0x7f302c1465f0 2023-09-29T23:29:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302a7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c0a9c70 0x7f302c14daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302a7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c0a9c70 0x7f302c14daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47876/0 (socket says 172.21.15.143:47876) 2023-09-29T23:29:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302a7fc700 1 -- 172.21.15.143:0/1502153574 learned_addr learned my addr 172.21.15.143:0/1502153574 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:40.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302b7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 0x7f302c14dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302affd700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f302c0a98a0 0x7f302c0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302a7fc700 1 -- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 msgr2=0x7f302c14dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302a7fc700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 0x7f302c14dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302a7fc700 1 -- 172.21.15.143:0/1502153574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f302c0a98a0 msgr2=0x7f302c0afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302a7fc700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f302c0a98a0 0x7f302c0afd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302a7fc700 1 -- 172.21.15.143:0/1502153574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3024009ae0 con 0x7f302c0a9c70 2023-09-29T23:29:40.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302a7fc700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c0a9c70 0x7f302c14daa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f301400ec80 tx=0x7f30140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f3022ffd700 1 -- 172.21.15.143:0/1502153574 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301400c850 con 0x7f302c0a9c70 2023-09-29T23:29:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f3022ffd700 1 -- 172.21.15.143:0/1502153574 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301400c9b0 con 0x7f302c0a9c70 2023-09-29T23:29:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302bfff700 1 -- 172.21.15.143:0/1502153574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c14e6e0 con 0x7f302c0a9c70 2023-09-29T23:29:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f3022ffd700 1 -- 172.21.15.143:0/1502153574 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301400cd50 con 0x7f302c0a9c70 2023-09-29T23:29:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302bfff700 1 -- 172.21.15.143:0/1502153574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c13d150 con 0x7f302c0a9c70 2023-09-29T23:29:40.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.020+0000 7f302bfff700 1 -- 172.21.15.143:0/1502153574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302c05ecb0 con 0x7f302c0a9c70 2023-09-29T23:29:40.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.024+0000 7f3022ffd700 1 -- 172.21.15.143:0/1502153574 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f3014013680 con 0x7f302c0a9c70 2023-09-29T23:29:40.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.024+0000 7f3022ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:40.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.024+0000 7f3022ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:40.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.024+0000 7f3022ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:40.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.024+0000 7f3022ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:40.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.024+0000 7f3022ffd700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f300c08e830 0x7f300c090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.024+0000 7f3022ffd700 1 -- 172.21.15.143:0/1502153574 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f30140b5310 con 0x7f302c0a9c70 2023-09-29T23:29:40.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.024+0000 7f302affd700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f300c08e830 0x7f300c090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.024+0000 7f302affd700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f300c08e830 0x7f300c090ce0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f301c0039e0 tx=0x7f301c007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:40.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.028+0000 7f3022ffd700 1 -- 172.21.15.143:0/1502153574 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f3014085ca0 con 0x7f302c0a9c70 2023-09-29T23:29:40.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.188+0000 7f302bfff700 1 -- 172.21.15.143:0/1502153574 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f302c141b60 con 0x7f302c0a9c70 2023-09-29T23:29:40.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f3022ffd700 1 -- 172.21.15.143:0/1502153574 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12219 (secure 0 0 0) 0x7f3014021090 con 0x7f302c0a9c70 2023-09-29T23:29:40.194 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:40.195 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":18,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:29:39.586692+0000","last_up_change":"2023-09-29T23:29:27.340059+0000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T23:29:28.520729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T23:29:33.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-29T23:29:37.927418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-29T23:29:38.594861+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":27741},{"type":"v1","addr":"172.21.15.143:6801","nonce":27741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":27741},{"type":"v1","addr":"172.21.15.143:6803","nonce":27741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":27741},{"type":"v1","addr":"172.21.15.143:6807","nonce":27741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":27741},{"type":"v1","addr":"172.21.15.143:6805","nonce":27741}]},"public_addr":"172.21.15.143:6801/27741","cluster_addr":"172.21.15.143:6803/27741","heartbeat_back_addr":"172.21.15.143:6807/27741","heartbeat_front_addr":"172.21.15.143:6805/27741","state":["exists","up"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":30036},{"type":"v1","addr":"172.21.15.156:6817","nonce":30036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":30036},{"type":"v1","addr":"172.21.15.156:6819","nonce":30036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":30036},{"type":"v1","addr":"172.21.15.156:6823","nonce":30036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":30036},{"type":"v1","addr":"172.21.15.156:6821","nonce":30036}]},"public_addr":"172.21.15.156:6817/30036","cluster_addr":"172.21.15.156:6819/30036","heartbeat_back_addr":"172.21.15.156:6823/30036","heartbeat_front_addr":"172.21.15.156:6821/30036","state":["exists","up"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":30035},{"type":"v1","addr":"172.21.15.156:6809","nonce":30035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":30035},{"type":"v1","addr":"172.21.15.156:6811","nonce":30035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":30035},{"type":"v1","addr":"172.21.15.156:6815","nonce":30035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":30035},{"type":"v1","addr":"172.21.15.156:6813","nonce":30035}]},"public_addr":"172.21.15.156:6809/30035","cluster_addr":"172.21.15.156:6811/30035","heartbeat_back_addr":"172.21.15.156:6815/30035","heartbeat_front_addr":"172.21.15.156:6813/30035","state":["exists","up"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":30037},{"type":"v1","addr":"172.21.15.156:6801","nonce":30037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":30037},{"type":"v1","addr":"172.21.15.156:6803","nonce":30037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":30037},{"type":"v1","addr":"172.21.15.156:6807","nonce":30037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":30037},{"type":"v1","addr":"172.21.15.156:6805","nonce":30037}]},"public_addr":"172.21.15.156:6801/30037","cluster_addr":"172.21.15.156:6803/30037","heartbeat_back_addr":"172.21.15.156:6807/30037","heartbeat_front_addr":"172.21.15.156:6805/30037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.781977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.943955+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.806513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.624301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:29:40.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 10 mgrc shutdown 2023-09-29T23:29:40.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 -- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f300c08e830 msgr2=0x7f300c090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f300c08e830 0x7f300c090ce0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f301c0039e0 tx=0x7f301c007ec0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 -- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c0a9c70 msgr2=0x7f302c14daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c0a9c70 0x7f302c14daa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f301400ec80 tx=0x7f30140091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 -- 172.21.15.143:0/1502153574 shutdown_connections 2023-09-29T23:29:40.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f300c08e830 0x7f300c090ce0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f302c1465f0 0x7f302c14dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f302c0a9c70 0x7f302c14daa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 --2- 172.21.15.143:0/1502153574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f302c0a98a0 0x7f302c0afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 -- 172.21.15.143:0/1502153574 >> 172.21.15.143:0/1502153574 conn(0x7f302c0b4730 msgr2=0x7f302c13d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:40.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 -- 172.21.15.143:0/1502153574 shutdown_connections 2023-09-29T23:29:40.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.192+0000 7f302bfff700 1 -- 172.21.15.143:0/1502153574 wait complete. 2023-09-29T23:29:40.209 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-29T23:29:40.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc955393700 1 Processor -- start 2023-09-29T23:29:40.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc955393700 1 -- start start 2023-09-29T23:29:40.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc955393700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc95013fe30 0x7fc9500a3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc955393700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9500a54a0 con 0x7fc9500a4220 2023-09-29T23:29:40.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc955393700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9500a4640 con 0x7fc9500673f0 2023-09-29T23:29:40.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc955393700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9500677c0 con 0x7fc95013fe30 2023-09-29T23:29:40.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc94f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc95013fe30 0x7fc9500a3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc94e7fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fc9500a4220 0x7fc95013f720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54844/0 (socket says 172.21.15.143:54844) 2023-09-29T23:29:40.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc94e7fc700 1 -- 172.21.15.143:0/1838498898 learned_addr learned my addr 172.21.15.143:0/1838498898 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:40.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc94dffb700 1 -- 172.21.15.143:0/1838498898 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551075726 0 0) 0x7fc9500a4640 con 0x7fc9500673f0 2023-09-29T23:29:40.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc94dffb700 1 -- 172.21.15.143:0/1838498898 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc93c003570 con 0x7fc9500673f0 2023-09-29T23:29:40.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc94dffb700 1 -- 172.21.15.143:0/1838498898 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613082235 0 0) 0x7fc9500a54a0 con 0x7fc9500a4220 2023-09-29T23:29:40.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc94f7fe700 1 -- 172.21.15.143:0/1838498898 >> v1:172.21.15.156:6789/0 conn(0x7fc9500673f0 legacy=0x7fc9500a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc94f7fe700 1 -- 172.21.15.143:0/1838498898 >> v1:172.21.15.143:6789/0 conn(0x7fc9500a4220 legacy=0x7fc95013f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.320+0000 7fc94f7fe700 1 -- 172.21.15.143:0/1838498898 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9500a56f0 con 0x7fc95013fe30 2023-09-29T23:29:40.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc94f7fe700 1 --2- 172.21.15.143:0/1838498898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95013fe30 0x7fc9500a3290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc9400098e0 tx=0x7fc9400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761b453f9853443f server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:40.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc94dffb700 1 -- 172.21.15.143:0/1838498898 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94000b640 con 0x7fc95013fe30 2023-09-29T23:29:40.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc94dffb700 1 -- 172.21.15.143:0/1838498898 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94000b7a0 con 0x7fc95013fe30 2023-09-29T23:29:40.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc94dffb700 1 -- 172.21.15.143:0/1838498898 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94000bb40 con 0x7fc95013fe30 2023-09-29T23:29:40.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc955393700 1 -- 172.21.15.143:0/1838498898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95013fe30 msgr2=0x7fc9500a3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc955393700 1 --2- 172.21.15.143:0/1838498898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95013fe30 0x7fc9500a3290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc9400098e0 tx=0x7fc9400091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc955393700 1 -- 172.21.15.143:0/1838498898 shutdown_connections 2023-09-29T23:29:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc955393700 1 --2- 172.21.15.143:0/1838498898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc95013fe30 0x7fc9500a3290 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc955393700 1 -- 172.21.15.143:0/1838498898 >> 172.21.15.143:0/1838498898 conn(0x7fc950069b90 msgr2=0x7fc950069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc955393700 1 -- 172.21.15.143:0/1838498898 shutdown_connections 2023-09-29T23:29:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc955393700 1 -- 172.21.15.143:0/1838498898 wait complete. 2023-09-29T23:29:40.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc955393700 1 Processor -- start 2023-09-29T23:29:40.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.324+0000 7fc955393700 1 -- start start 2023-09-29T23:29:40.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc955393700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9500673f0 0x7fc95006c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc955393700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc9500a4220 0x7fc95013bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc955393700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc95013fe30 0x7fc95013c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc955393700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95013c990 con 0x7fc9500673f0 2023-09-29T23:29:40.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc955393700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95013cc50 con 0x7fc95013fe30 2023-09-29T23:29:40.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc955393700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95006cac0 con 0x7fc9500a4220 2023-09-29T23:29:40.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94e7fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc9500a4220 0x7fc95013bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9500673f0 0x7fc95006c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9500673f0 0x7fc95006c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47892/0 (socket says 172.21.15.143:47892) 2023-09-29T23:29:40.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94e7fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc9500a4220 0x7fc95013bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60386/0 (socket says 172.21.15.143:60386) 2023-09-29T23:29:40.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94effd700 1 -- 172.21.15.143:0/3055650647 learned_addr learned my addr 172.21.15.143:0/3055650647 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:40.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94f7fe700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc95013fe30 0x7fc95013c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94effd700 1 -- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc9500a4220 msgr2=0x7fc95013bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94effd700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc9500a4220 0x7fc95013bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94effd700 1 -- 172.21.15.143:0/3055650647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc95013fe30 msgr2=0x7fc95013c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94effd700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc95013fe30 0x7fc95013c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94effd700 1 -- 172.21.15.143:0/3055650647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc940009420 con 0x7fc9500673f0 2023-09-29T23:29:40.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc94effd700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9500673f0 0x7fc95006c580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc94800cb10 tx=0x7fc948002820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:40.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc946ffd700 1 -- 172.21.15.143:0/3055650647 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800a7e0 con 0x7fc9500673f0 2023-09-29T23:29:40.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc946ffd700 1 -- 172.21.15.143:0/3055650647 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94800a940 con 0x7fc9500673f0 2023-09-29T23:29:40.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc955393700 1 -- 172.21.15.143:0/3055650647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95006de10 con 0x7fc9500673f0 2023-09-29T23:29:40.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc946ffd700 1 -- 172.21.15.143:0/3055650647 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94800ad00 con 0x7fc9500673f0 2023-09-29T23:29:40.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc955393700 1 -- 172.21.15.143:0/3055650647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95006e9b0 con 0x7fc9500673f0 2023-09-29T23:29:40.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.328+0000 7fc955393700 1 -- 172.21.15.143:0/3055650647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95005ecb0 con 0x7fc9500673f0 2023-09-29T23:29:40.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.332+0000 7fc946ffd700 1 -- 172.21.15.143:0/3055650647 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fc948012630 con 0x7fc9500673f0 2023-09-29T23:29:40.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.332+0000 7fc946ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:40.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.332+0000 7fc946ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:40.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.332+0000 7fc946ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:40.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.332+0000 7fc946ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:40.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.332+0000 7fc946ffd700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc93008e860 0x7fc930090d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.332+0000 7fc94e7fc700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc93008e860 0x7fc930090d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.332+0000 7fc946ffd700 1 -- 172.21.15.143:0/3055650647 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4330+0+0 (secure 0 0 0) 0x7fc9480b4a40 con 0x7fc9500673f0 2023-09-29T23:29:40.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.332+0000 7fc94e7fc700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc93008e860 0x7fc930090d10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc95006d310 tx=0x7fc938007d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:40.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.340+0000 7fc946ffd700 1 -- 172.21.15.143:0/3055650647 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc9480b6e60 con 0x7fc9500673f0 2023-09-29T23:29:40.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.536+0000 7fc955393700 1 -- 172.21.15.143:0/3055650647 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc9500a59b0 con 0x7fc9500673f0 2023-09-29T23:29:40.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.600+0000 7fc946ffd700 1 -- 172.21.15.143:0/3055650647 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc948085500 con 0x7fc9500673f0 2023-09-29T23:29:40.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 10 mgrc shutdown 2023-09-29T23:29:40.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 -- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc93008e860 msgr2=0x7fc930090d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc93008e860 0x7fc930090d10 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc95006d310 tx=0x7fc938007d10 comp rx=0 tx=0).stop 2023-09-29T23:29:40.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 -- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9500673f0 msgr2=0x7fc95006c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9500673f0 0x7fc95006c580 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc94800cb10 tx=0x7fc948002820 comp rx=0 tx=0).stop 2023-09-29T23:29:40.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 -- 172.21.15.143:0/3055650647 shutdown_connections 2023-09-29T23:29:40.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc93008e860 0x7fc930090d10 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc95013fe30 0x7fc95013c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc9500a4220 0x7fc95013bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 --2- 172.21.15.143:0/3055650647 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9500673f0 0x7fc95006c580 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 -- 172.21.15.143:0/3055650647 >> 172.21.15.143:0/3055650647 conn(0x7fc950069b90 msgr2=0x7fc9500b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:40.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 -- 172.21.15.143:0/3055650647 shutdown_connections 2023-09-29T23:29:40.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.604+0000 7fc955393700 1 -- 172.21.15.143:0/3055650647 wait complete. 2023-09-29T23:29:40.622 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-29T23:29:40.625 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-29T23:29:40.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.736+0000 7ff5ae5d1700 1 Processor -- start 2023-09-29T23:29:40.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.736+0000 7ff5ae5d1700 1 -- start start 2023-09-29T23:29:40.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.736+0000 7ff5ae5d1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff5a80a6ea0 0x7ff5a80a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.736+0000 7ff5ae5d1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff5a80a7270 0x7ff5a806cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.736+0000 7ff5ae5d1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a5fd0 0x7ff5a806d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.736+0000 7ff5ae5d1700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a806da40 con 0x7ff5a80a6ea0 2023-09-29T23:29:40.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.736+0000 7ff5ae5d1700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a806e7a0 con 0x7ff5a80a7270 2023-09-29T23:29:40.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.736+0000 7ff5ae5d1700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a806dc50 con 0x7ff5a80a5fd0 2023-09-29T23:29:40.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5addd0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a5fd0 0x7ff5a806d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ad5cf700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ff5a80a6ea0 0x7ff5a80a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5addd0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a5fd0 0x7ff5a806d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60400/0 (socket says 172.21.15.143:60400) 2023-09-29T23:29:40.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5addd0700 1 -- 172.21.15.143:0/1517749934 learned_addr learned my addr 172.21.15.143:0/1517749934 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:40.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5acdce700 1 --2- 172.21.15.143:0/1517749934 >> v2:172.21.15.156:3300/0 conn(0x7ff5a80a7270 0x7ff5a806cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5addd0700 1 -- 172.21.15.143:0/1517749934 >> v2:172.21.15.156:3300/0 conn(0x7ff5a80a7270 msgr2=0x7ff5a806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:40.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5addd0700 1 --2- 172.21.15.143:0/1517749934 >> v2:172.21.15.156:3300/0 conn(0x7ff5a80a7270 0x7ff5a806cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5addd0700 1 -- 172.21.15.143:0/1517749934 >> v2:172.21.15.143:3300/0 conn(0x7ff5a80a6ea0 msgr2=0x7ff5a80a5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:40.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5addd0700 1 --2- 172.21.15.143:0/1517749934 >> v2:172.21.15.143:3300/0 conn(0x7ff5a80a6ea0 0x7ff5a80a5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5addd0700 1 -- 172.21.15.143:0/1517749934 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a806df10 con 0x7ff5a80a5fd0 2023-09-29T23:29:40.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5addd0700 1 --2- 172.21.15.143:0/1517749934 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a5fd0 0x7ff5a806d500 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff5a806e100 tx=0x7ff5a0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbf442e072d79817 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:40.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff59ffff700 1 -- 172.21.15.143:0/1517749934 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000e070 con 0x7ff5a80a5fd0 2023-09-29T23:29:40.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff59ffff700 1 -- 172.21.15.143:0/1517749934 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a000b9f0 con 0x7ff5a80a5fd0 2023-09-29T23:29:40.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff59ffff700 1 -- 172.21.15.143:0/1517749934 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000bdb0 con 0x7ff5a80a5fd0 2023-09-29T23:29:40.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/1517749934 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a5fd0 msgr2=0x7ff5a806d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ae5d1700 1 --2- 172.21.15.143:0/1517749934 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a5fd0 0x7ff5a806d500 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff5a806e100 tx=0x7ff5a0009840 comp rx=0 tx=0).stop 2023-09-29T23:29:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/1517749934 shutdown_connections 2023-09-29T23:29:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ae5d1700 1 --2- 172.21.15.143:0/1517749934 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a5fd0 0x7ff5a806d500 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ae5d1700 1 --2- 172.21.15.143:0/1517749934 >> v2:172.21.15.156:3300/0 conn(0x7ff5a80a7270 0x7ff5a806cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ae5d1700 1 --2- 172.21.15.143:0/1517749934 >> v2:172.21.15.143:3300/0 conn(0x7ff5a80a6ea0 0x7ff5a80a5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/1517749934 >> 172.21.15.143:0/1517749934 conn(0x7ff5a80b4560 msgr2=0x7ff5a80b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/1517749934 shutdown_connections 2023-09-29T23:29:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/1517749934 wait complete. 2023-09-29T23:29:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ae5d1700 1 Processor -- start 2023-09-29T23:29:40.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.740+0000 7ff5ae5d1700 1 -- start start 2023-09-29T23:29:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ae5d1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5a80a5fd0 0x7ff5a813bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ae5d1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5a80a6ea0 0x7ff5a81515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ae5d1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a7270 0x7ff5a8151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ae5d1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a8151090 con 0x7ff5a80a5fd0 2023-09-29T23:29:40.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ae5d1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a8152060 con 0x7ff5a80a6ea0 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ae5d1700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a81394a0 con 0x7ff5a80a7270 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ad5cf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5a80a5fd0 0x7ff5a813bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5addd0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a7270 0x7ff5a8151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ad5cf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5a80a5fd0 0x7ff5a813bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47918/0 (socket says 172.21.15.143:47918) 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ad5cf700 1 -- 172.21.15.143:0/3066979474 learned_addr learned my addr 172.21.15.143:0/3066979474 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5acdce700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5a80a6ea0 0x7ff5a81515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5addd0700 1 -- 172.21.15.143:0/3066979474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5a80a6ea0 msgr2=0x7ff5a81515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5addd0700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5a80a6ea0 0x7ff5a81515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5addd0700 1 -- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5a80a5fd0 msgr2=0x7ff5a813bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5addd0700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5a80a5fd0 0x7ff5a813bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5addd0700 1 -- 172.21.15.143:0/3066979474 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a0009ae0 con 0x7ff5a80a7270 2023-09-29T23:29:40.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ad5cf700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5a80a5fd0 0x7ff5a813bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5acdce700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5a80a6ea0 0x7ff5a81515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5addd0700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a7270 0x7ff5a8151b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0014000 tx=0x7ff5a0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:40.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff59d7fa700 1 -- 172.21.15.143:0/3066979474 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a000bee0 con 0x7ff5a80a7270 2023-09-29T23:29:40.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/3066979474 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a81391b0 con 0x7ff5a80a7270 2023-09-29T23:29:40.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/3066979474 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a8140af0 con 0x7ff5a80a7270 2023-09-29T23:29:40.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff59d7fa700 1 -- 172.21.15.143:0/3066979474 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a00032f0 con 0x7ff5a80a7270 2023-09-29T23:29:40.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.744+0000 7ff59d7fa700 1 -- 172.21.15.143:0/3066979474 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0009940 con 0x7ff5a80a7270 2023-09-29T23:29:40.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.748+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/3066979474 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5880050b0 con 0x7ff5a80a7270 2023-09-29T23:29:40.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.752+0000 7ff59d7fa700 1 -- 172.21.15.143:0/3066979474 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7ff5a0008030 con 0x7ff5a80a7270 2023-09-29T23:29:40.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.752+0000 7ff59d7fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:40.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.752+0000 7ff59d7fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:40.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.752+0000 7ff59d7fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.752+0000 7ff59d7fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.752+0000 7ff59d7fa700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff58c08e830 0x7ff58c090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.752+0000 7ff5ad5cf700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff58c08e830 0x7ff58c090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.752+0000 7ff59d7fa700 1 -- 172.21.15.143:0/3066979474 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7ff5a00b6390 con 0x7ff5a80a7270 2023-09-29T23:29:40.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.752+0000 7ff5ad5cf700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff58c08e830 0x7ff58c090ce0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5980097e0 tx=0x7ff598012040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:40.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.756+0000 7ff59d7fa700 1 -- 172.21.15.143:0/3066979474 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff5a0086db0 con 0x7ff5a80a7270 2023-09-29T23:29:40.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:40.952+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/3066979474 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff588005630 con 0x7ff5a80a7270 2023-09-29T23:29:41.224 INFO:tasks.ceph.mds.c.smithi143.stderr:2023-09-29T23:29:41.220+0000 7f8c33be5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-29T23:29:41.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.608+0000 7ff59d7fa700 1 -- 172.21.15.143:0/3066979474 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff5a008a3d0 con 0x7ff5a80a7270 2023-09-29T23:29:41.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 10 mgrc shutdown 2023-09-29T23:29:41.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff58c08e830 msgr2=0x7ff58c090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:41.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff58c08e830 0x7ff58c090ce0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff5980097e0 tx=0x7ff598012040 comp rx=0 tx=0).stop 2023-09-29T23:29:41.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a7270 msgr2=0x7ff5a8151b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:41.616 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a7270 0x7ff5a8151b20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0014000 tx=0x7ff5a0017000 comp rx=0 tx=0).stop 2023-09-29T23:29:41.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/3066979474 shutdown_connections 2023-09-29T23:29:41.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff58c08e830 0x7ff58c090ce0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5a80a7270 0x7ff5a8151b20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5a80a6ea0 0x7ff5a81515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 --2- 172.21.15.143:0/3066979474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5a80a5fd0 0x7ff5a813bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/3066979474 >> 172.21.15.143:0/3066979474 conn(0x7ff5a80b4560 msgr2=0x7ff5a806cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:41.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/3066979474 shutdown_connections 2023-09-29T23:29:41.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.612+0000 7ff5ae5d1700 1 -- 172.21.15.143:0/3066979474 wait complete. 2023-09-29T23:29:41.634 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-29T23:29:41.634 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-29T23:29:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.744+0000 7f8f11b57700 1 Processor -- start 2023-09-29T23:29:41.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.744+0000 7f8f11b57700 1 -- start start 2023-09-29T23:29:41.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.744+0000 7f8f11b57700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8f0c066ec0 0x7f8f0c0a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:41.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.744+0000 7f8f11b57700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8f0c09f830 0x7f8f0c13d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:41.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.744+0000 7f8f11b57700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0c0a16b0 con 0x7f8f0c09fc40 2023-09-29T23:29:41.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.744+0000 7f8f11b57700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c067290 con 0x7f8f0c09f830 2023-09-29T23:29:41.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.744+0000 7f8f11b57700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0a0510 con 0x7f8f0c066ec0 2023-09-29T23:29:41.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.744+0000 7f8f10b55700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8f0c066ec0 0x7f8f0c0a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:41.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.744+0000 7f8f11356700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8f0c09fc40 0x7f8f0c141220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54860/0 (socket says 172.21.15.143:54860) 2023-09-29T23:29:41.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.744+0000 7f8f11356700 1 -- 172.21.15.143:0/3625187327 learned_addr learned my addr 172.21.15.143:0/3625187327 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:41.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f0bfff700 1 --2- 172.21.15.143:0/3625187327 >> v2:172.21.15.156:3300/0 conn(0x7f8f0c09f830 0x7f8f0c13d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:41.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f0b7fe700 1 -- 172.21.15.143:0/3625187327 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987766437 0 0) 0x7f8f0c0a16b0 con 0x7f8f0c09fc40 2023-09-29T23:29:41.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f0b7fe700 1 -- 172.21.15.143:0/3625187327 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ef8003570 con 0x7f8f0c09fc40 2023-09-29T23:29:41.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f10b55700 1 -- 172.21.15.143:0/3625187327 >> v2:172.21.15.156:3300/0 conn(0x7f8f0c09f830 msgr2=0x7f8f0c13d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:41.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f10b55700 1 --2- 172.21.15.143:0/3625187327 >> v2:172.21.15.156:3300/0 conn(0x7f8f0c09f830 0x7f8f0c13d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f10b55700 1 -- 172.21.15.143:0/3625187327 >> v1:172.21.15.143:6789/0 conn(0x7f8f0c09fc40 legacy=0x7f8f0c141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:41.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f10b55700 1 -- 172.21.15.143:0/3625187327 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c06e400 con 0x7f8f0c066ec0 2023-09-29T23:29:41.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f10b55700 1 --2- 172.21.15.143:0/3625187327 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8f0c066ec0 0x7f8f0c0a1170 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40098e0 tx=0x7f8ef40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd2703382418aed server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:41.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f0b7fe700 1 -- 172.21.15.143:0/3625187327 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4011630 con 0x7f8f0c066ec0 2023-09-29T23:29:41.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f0b7fe700 1 -- 172.21.15.143:0/3625187327 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef4011790 con 0x7f8f0c066ec0 2023-09-29T23:29:41.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f0b7fe700 1 -- 172.21.15.143:0/3625187327 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4011b30 con 0x7f8f0c066ec0 2023-09-29T23:29:41.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f11b57700 1 -- 172.21.15.143:0/3625187327 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8f0c066ec0 msgr2=0x7f8f0c0a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:41.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f11b57700 1 --2- 172.21.15.143:0/3625187327 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8f0c066ec0 0x7f8f0c0a1170 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40098e0 tx=0x7f8ef40091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f11b57700 1 -- 172.21.15.143:0/3625187327 shutdown_connections 2023-09-29T23:29:41.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f11b57700 1 --2- 172.21.15.143:0/3625187327 >> v2:172.21.15.156:3300/0 conn(0x7f8f0c09f830 0x7f8f0c13d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f11b57700 1 --2- 172.21.15.143:0/3625187327 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8f0c066ec0 0x7f8f0c0a1170 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f11b57700 1 -- 172.21.15.143:0/3625187327 >> 172.21.15.143:0/3625187327 conn(0x7f8f0c06f200 msgr2=0x7f8f0c069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:41.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f11b57700 1 -- 172.21.15.143:0/3625187327 shutdown_connections 2023-09-29T23:29:41.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f11b57700 1 -- 172.21.15.143:0/3625187327 wait complete. 2023-09-29T23:29:41.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f11b57700 1 Processor -- start 2023-09-29T23:29:41.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.748+0000 7f8f11b57700 1 -- start start 2023-09-29T23:29:41.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f11b57700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8f0c066ec0 0x7f8f0c09a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:41.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f11b57700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f0c09f830 0x7f8f0c137600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:41.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f11b57700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f0c09fc40 0x7f8f0c137b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:41.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f11b57700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c138080 con 0x7f8f0c09f830 2023-09-29T23:29:41.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f11b57700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c137170 con 0x7f8f0c09fc40 2023-09-29T23:29:41.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f11b57700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c06abe0 con 0x7f8f0c066ec0 2023-09-29T23:29:41.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f0bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f0c09f830 0x7f8f0c137600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:41.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f10b55700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8f0c066ec0 0x7f8f0c09a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:41.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f0bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f0c09f830 0x7f8f0c137600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47930/0 (socket says 172.21.15.143:47930) 2023-09-29T23:29:41.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f0bfff700 1 -- 172.21.15.143:0/1983260712 learned_addr learned my addr 172.21.15.143:0/1983260712 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:41.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f11356700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f0c09fc40 0x7f8f0c137b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:41.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f0bfff700 1 -- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8f0c066ec0 msgr2=0x7f8f0c09a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:41.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f0bfff700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8f0c066ec0 0x7f8f0c09a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f0bfff700 1 -- 172.21.15.143:0/1983260712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f0c09fc40 msgr2=0x7f8f0c137b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:41.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f0bfff700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f0c09fc40 0x7f8f0c137b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f0bfff700 1 -- 172.21.15.143:0/1983260712 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef4009420 con 0x7f8f0c09f830 2023-09-29T23:29:41.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f0bfff700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f0c09f830 0x7f8f0c137600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009f20 tx=0x7f8efc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:41.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f08ff9700 1 -- 172.21.15.143:0/1983260712 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc00cbd0 con 0x7f8f0c09f830 2023-09-29T23:29:41.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f08ff9700 1 -- 172.21.15.143:0/1983260712 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc00f040 con 0x7f8f0c09f830 2023-09-29T23:29:41.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f11b57700 1 -- 172.21.15.143:0/1983260712 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c143170 con 0x7f8f0c09f830 2023-09-29T23:29:41.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f08ff9700 1 -- 172.21.15.143:0/1983260712 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc014480 con 0x7f8f0c09f830 2023-09-29T23:29:41.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f11b57700 1 -- 172.21.15.143:0/1983260712 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c09b110 con 0x7f8f0c09f830 2023-09-29T23:29:41.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.752+0000 7f8f11b57700 1 -- 172.21.15.143:0/1983260712 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c05ecb0 con 0x7f8f0c09f830 2023-09-29T23:29:41.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.756+0000 7f8f08ff9700 1 -- 172.21.15.143:0/1983260712 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f8efc0147f0 con 0x7f8f0c09f830 2023-09-29T23:29:41.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.756+0000 7f8f08ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:41.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.756+0000 7f8f08ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:41.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.756+0000 7f8f08ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:41.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.756+0000 7f8f08ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:41.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.756+0000 7f8f08ff9700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8eec08e8e0 0x7f8eec090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:41.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.756+0000 7f8f10b55700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8eec08e8e0 0x7f8eec090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:41.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.756+0000 7f8f08ff9700 1 -- 172.21.15.143:0/1983260712 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f8efc0b51b0 con 0x7f8f0c09f830 2023-09-29T23:29:41.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.756+0000 7f8f10b55700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8eec08e8e0 0x7f8eec090d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40091e0 tx=0x7f8ef4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:41.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.764+0000 7f8f08ff9700 1 -- 172.21.15.143:0/1983260712 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8efc085be0 con 0x7f8f0c09f830 2023-09-29T23:29:41.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.956+0000 7f8f11b57700 1 -- 172.21.15.143:0/1983260712 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f0c06e6c0 con 0x7f8f0c09f830 2023-09-29T23:29:41.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.956+0000 7f8f08ff9700 1 -- 172.21.15.143:0/1983260712 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3841 (secure 0 0 0) 0x7f8efc089200 con 0x7f8f0c09f830 2023-09-29T23:29:41.960 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:41.960 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4214,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6825/2413469850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2413469850},{"type":"v1","addr":"172.21.15.156:6825","nonce":2413469850}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4233,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6811/896631119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":896631119},{"type":"v1","addr":"172.21.15.143:6811","nonce":896631119}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-29T23:29:39.587347+0000","modified":"2023-09-29T23:29:41.602676+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4258},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4258":{"gid":4258,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.143:6813/1935385308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1935385308},{"type":"v1","addr":"172.21.15.143:6813","nonce":1935385308}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-29T23:29:41.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 10 mgrc shutdown 2023-09-29T23:29:41.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 -- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8eec08e8e0 msgr2=0x7f8eec090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:41.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8eec08e8e0 0x7f8eec090d90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ef40091e0 tx=0x7f8ef4017000 comp rx=0 tx=0).stop 2023-09-29T23:29:41.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 -- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f0c09f830 msgr2=0x7f8f0c137600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:41.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f0c09f830 0x7f8f0c137600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8efc009f20 tx=0x7f8efc00b040 comp rx=0 tx=0).stop 2023-09-29T23:29:41.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 -- 172.21.15.143:0/1983260712 shutdown_connections 2023-09-29T23:29:41.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8eec08e8e0 0x7f8eec090d90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f0c09fc40 0x7f8f0c137b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f0c09f830 0x7f8f0c137600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 --2- 172.21.15.143:0/1983260712 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8f0c066ec0 0x7f8f0c09a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:41.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 -- 172.21.15.143:0/1983260712 >> 172.21.15.143:0/1983260712 conn(0x7f8f0c06f200 msgr2=0x7f8f0c13a460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:41.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 -- 172.21.15.143:0/1983260712 shutdown_connections 2023-09-29T23:29:41.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:41.960+0000 7f8f11b57700 1 -- 172.21.15.143:0/1983260712 wait complete. 2023-09-29T23:29:41.964 INFO:teuthology.orchestra.run.smithi143.stderr:dumped fsmap epoch 6 2023-09-29T23:29:41.973 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T23:29:42.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95d9623700 1 Processor -- start 2023-09-29T23:29:42.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95d9623700 1 -- start start 2023-09-29T23:29:42.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95d9623700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f95d40a5750 0x7f95d40a4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95d9623700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d406e1d0 con 0x7f95d406de00 2023-09-29T23:29:42.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95d9623700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d4069c40 con 0x7f95d4069ee0 2023-09-29T23:29:42.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95d9623700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d4069820 con 0x7f95d40a5750 2023-09-29T23:29:42.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95d3fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f95d406de00 0x7f95d40a6940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54868/0 (socket says 172.21.15.143:54868) 2023-09-29T23:29:42.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95d3fff700 1 -- 172.21.15.143:0/1301404716 learned_addr learned my addr 172.21.15.143:0/1301404716 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:42.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95cb7fe700 1 --2- 172.21.15.143:0/1301404716 >> v2:172.21.15.143:3300/0 conn(0x7f95d40a5750 0x7f95d40a4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95cb7fe700 1 -- 172.21.15.143:0/1301404716 >> v1:172.21.15.156:6789/0 conn(0x7f95d4069ee0 legacy=0x7f95d414a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95cb7fe700 1 -- 172.21.15.143:0/1301404716 >> v1:172.21.15.143:6789/0 conn(0x7f95d406de00 legacy=0x7f95d40a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95cb7fe700 1 -- 172.21.15.143:0/1301404716 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d4069960 con 0x7f95d40a5750 2023-09-29T23:29:42.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.076+0000 7f95cb7fe700 1 --2- 172.21.15.143:0/1301404716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95d40a5750 0x7f95d40a4bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009940 tx=0x7f95bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3bf1d82c162015 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:42.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d37fe700 1 -- 172.21.15.143:0/1301404716 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00b7e0 con 0x7f95d40a5750 2023-09-29T23:29:42.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d37fe700 1 -- 172.21.15.143:0/1301404716 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc00b940 con 0x7f95d40a5750 2023-09-29T23:29:42.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d37fe700 1 -- 172.21.15.143:0/1301404716 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00bce0 con 0x7f95d40a5750 2023-09-29T23:29:42.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d9623700 1 -- 172.21.15.143:0/1301404716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95d40a5750 msgr2=0x7f95d40a4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d9623700 1 --2- 172.21.15.143:0/1301404716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95d40a5750 0x7f95d40a4bf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009940 tx=0x7f95bc0091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d9623700 1 -- 172.21.15.143:0/1301404716 shutdown_connections 2023-09-29T23:29:42.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d9623700 1 --2- 172.21.15.143:0/1301404716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95d40a5750 0x7f95d40a4bf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d9623700 1 -- 172.21.15.143:0/1301404716 >> 172.21.15.143:0/1301404716 conn(0x7f95d40b4640 msgr2=0x7f95d40b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:42.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d9623700 1 -- 172.21.15.143:0/1301404716 shutdown_connections 2023-09-29T23:29:42.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d9623700 1 -- 172.21.15.143:0/1301404716 wait complete. 2023-09-29T23:29:42.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d9623700 1 Processor -- start 2023-09-29T23:29:42.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.080+0000 7f95d9623700 1 -- start start 2023-09-29T23:29:42.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d9623700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95d4069ee0 0x7f95d409d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d9623700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f95d406de00 0x7f95d409ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d9623700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95d40a5750 0x7f95d414fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d9623700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d40a8e70 con 0x7f95d40a5750 2023-09-29T23:29:42.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d9623700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d4150a20 con 0x7f95d4069ee0 2023-09-29T23:29:42.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d9623700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d4150ce0 con 0x7f95d406de00 2023-09-29T23:29:42.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95cb7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f95d406de00 0x7f95d409ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d8e22700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95d40a5750 0x7f95d414fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95cb7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f95d406de00 0x7f95d409ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60432/0 (socket says 172.21.15.143:60432) 2023-09-29T23:29:42.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95cb7fe700 1 -- 172.21.15.143:0/1503640026 learned_addr learned my addr 172.21.15.143:0/1503640026 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:42.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d3fff700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95d4069ee0 0x7f95d409d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95cb7fe700 1 -- 172.21.15.143:0/1503640026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95d4069ee0 msgr2=0x7f95d409d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95cb7fe700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95d4069ee0 0x7f95d409d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95cb7fe700 1 -- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95d40a5750 msgr2=0x7f95d414fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95cb7fe700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95d40a5750 0x7f95d414fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95cb7fe700 1 -- 172.21.15.143:0/1503640026 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc009420 con 0x7f95d406de00 2023-09-29T23:29:42.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d8e22700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95d40a5750 0x7f95d414fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:42.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95cb7fe700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f95d406de00 0x7f95d409ae60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009fd0 tx=0x7f95bc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:42.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d0ff9700 1 -- 172.21.15.143:0/1503640026 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00bcc0 con 0x7f95d406de00 2023-09-29T23:29:42.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d0ff9700 1 -- 172.21.15.143:0/1503640026 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95bc024440 con 0x7f95d406de00 2023-09-29T23:29:42.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d9623700 1 -- 172.21.15.143:0/1503640026 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d409dac0 con 0x7f95d406de00 2023-09-29T23:29:42.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d0ff9700 1 -- 172.21.15.143:0/1503640026 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95bc024800 con 0x7f95d406de00 2023-09-29T23:29:42.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.084+0000 7f95d9623700 1 -- 172.21.15.143:0/1503640026 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d413fdf0 con 0x7f95d406de00 2023-09-29T23:29:42.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.088+0000 7f95d9623700 1 -- 172.21.15.143:0/1503640026 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d413ffe0 con 0x7f95d406de00 2023-09-29T23:29:42.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.088+0000 7f95d0ff9700 1 -- 172.21.15.143:0/1503640026 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f95bc02d440 con 0x7f95d406de00 2023-09-29T23:29:42.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.088+0000 7f95d0ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:42.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.088+0000 7f95d0ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:42.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.088+0000 7f95d0ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:42.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.088+0000 7f95d0ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:42.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.088+0000 7f95d0ff9700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f95b408e8b0 0x7f95b4090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.088+0000 7f95d0ff9700 1 -- 172.21.15.143:0/1503640026 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f95bc0b61c0 con 0x7f95d406de00 2023-09-29T23:29:42.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.088+0000 7f95d3fff700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f95b408e8b0 0x7f95b4090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.088+0000 7f95d3fff700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f95b408e8b0 0x7f95b4090d60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f95c4004fc0 tx=0x7f95c4007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:42.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.096+0000 7f95d0ff9700 1 -- 172.21.15.143:0/1503640026 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f95bc086c80 con 0x7f95d406de00 2023-09-29T23:29:42.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.252+0000 7f95d9623700 1 -- 172.21.15.143:0/1503640026 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95d405ecb0 con 0x7f95d406de00 2023-09-29T23:29:42.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.256+0000 7f95d0ff9700 1 -- 172.21.15.143:0/1503640026 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12206 (secure 0 0 0) 0x7f95bc08a2a0 con 0x7f95d406de00 2023-09-29T23:29:42.257 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:42.258 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:29:40.590094+0000","last_up_change":"2023-09-29T23:29:27.340059+0000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T23:29:28.520729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T23:29:33.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-29T23:29:37.927418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-29T23:29:38.594861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":27741},{"type":"v1","addr":"172.21.15.143:6801","nonce":27741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":27741},{"type":"v1","addr":"172.21.15.143:6803","nonce":27741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":27741},{"type":"v1","addr":"172.21.15.143:6807","nonce":27741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":27741},{"type":"v1","addr":"172.21.15.143:6805","nonce":27741}]},"public_addr":"172.21.15.143:6801/27741","cluster_addr":"172.21.15.143:6803/27741","heartbeat_back_addr":"172.21.15.143:6807/27741","heartbeat_front_addr":"172.21.15.143:6805/27741","state":["exists","up"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":30036},{"type":"v1","addr":"172.21.15.156:6817","nonce":30036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":30036},{"type":"v1","addr":"172.21.15.156:6819","nonce":30036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":30036},{"type":"v1","addr":"172.21.15.156:6823","nonce":30036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":30036},{"type":"v1","addr":"172.21.15.156:6821","nonce":30036}]},"public_addr":"172.21.15.156:6817/30036","cluster_addr":"172.21.15.156:6819/30036","heartbeat_back_addr":"172.21.15.156:6823/30036","heartbeat_front_addr":"172.21.15.156:6821/30036","state":["exists","up"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":30035},{"type":"v1","addr":"172.21.15.156:6809","nonce":30035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":30035},{"type":"v1","addr":"172.21.15.156:6811","nonce":30035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":30035},{"type":"v1","addr":"172.21.15.156:6815","nonce":30035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":30035},{"type":"v1","addr":"172.21.15.156:6813","nonce":30035}]},"public_addr":"172.21.15.156:6809/30035","cluster_addr":"172.21.15.156:6811/30035","heartbeat_back_addr":"172.21.15.156:6815/30035","heartbeat_front_addr":"172.21.15.156:6813/30035","state":["exists","up"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":30037},{"type":"v1","addr":"172.21.15.156:6801","nonce":30037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":30037},{"type":"v1","addr":"172.21.15.156:6803","nonce":30037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":30037},{"type":"v1","addr":"172.21.15.156:6807","nonce":30037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":30037},{"type":"v1","addr":"172.21.15.156:6805","nonce":30037}]},"public_addr":"172.21.15.156:6801/30037","cluster_addr":"172.21.15.156:6803/30037","heartbeat_back_addr":"172.21.15.156:6807/30037","heartbeat_front_addr":"172.21.15.156:6805/30037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.781977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.943955+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.806513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.624301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:29:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.256+0000 7f95d9623700 10 mgrc shutdown 2023-09-29T23:29:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.256+0000 7f95d9623700 1 -- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f95b408e8b0 msgr2=0x7f95b4090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.256+0000 7f95d9623700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f95b408e8b0 0x7f95b4090d60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f95c4004fc0 tx=0x7f95c4007eb0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.256+0000 7f95d9623700 1 -- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f95d406de00 msgr2=0x7f95d409ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.256+0000 7f95d9623700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f95d406de00 0x7f95d409ae60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009fd0 tx=0x7f95bc017000 comp rx=0 tx=0).stop 2023-09-29T23:29:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.256+0000 7f95d9623700 1 -- 172.21.15.143:0/1503640026 shutdown_connections 2023-09-29T23:29:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.256+0000 7f95d9623700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f95b408e8b0 0x7f95b4090d60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.256+0000 7f95d9623700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95d40a5750 0x7f95d414fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.256+0000 7f95d9623700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f95d406de00 0x7f95d409ae60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.256+0000 7f95d9623700 1 --2- 172.21.15.143:0/1503640026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95d4069ee0 0x7f95d409d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.260+0000 7f95d9623700 1 -- 172.21.15.143:0/1503640026 >> 172.21.15.143:0/1503640026 conn(0x7f95d40b4640 msgr2=0x7f95d4066e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.260+0000 7f95d9623700 1 -- 172.21.15.143:0/1503640026 shutdown_connections 2023-09-29T23:29:42.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.260+0000 7f95d9623700 1 -- 172.21.15.143:0/1503640026 wait complete. 2023-09-29T23:29:42.276 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-29T23:29:42.276 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T23:29:42.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.372+0000 7fabf892d700 1 Processor -- start 2023-09-29T23:29:42.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.372+0000 7fabf892d700 1 -- start start 2023-09-29T23:29:42.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.372+0000 7fabf892d700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fabf40a1170 0x7fabf413d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.372+0000 7fabf892d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf409ff40 con 0x7fabf413d680 2023-09-29T23:29:42.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.372+0000 7fabf892d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf4067290 con 0x7fabf4066ec0 2023-09-29T23:29:42.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.372+0000 7fabf892d700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf40a0510 con 0x7fabf40a1170 2023-09-29T23:29:42.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.372+0000 7fabf359e700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fabf413d680 0x7fabf4148590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54884/0 (socket says 172.21.15.143:54884) 2023-09-29T23:29:42.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.372+0000 7fabf359e700 1 -- 172.21.15.143:0/2186041269 learned_addr learned my addr 172.21.15.143:0/2186041269 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:42.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf259c700 1 --2- 172.21.15.143:0/2186041269 >> v2:172.21.15.156:3300/0 conn(0x7fabf40a1170 0x7fabf413d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf1d9b700 1 -- 172.21.15.143:0/2186041269 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679902249 0 0) 0x7fabf409ff40 con 0x7fabf413d680 2023-09-29T23:29:42.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf1d9b700 1 -- 172.21.15.143:0/2186041269 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabdc003570 con 0x7fabf413d680 2023-09-29T23:29:42.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf259c700 1 -- 172.21.15.143:0/2186041269 >> v1:172.21.15.156:6789/0 conn(0x7fabf4066ec0 legacy=0x7fabf409f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf259c700 1 -- 172.21.15.143:0/2186041269 >> v1:172.21.15.143:6789/0 conn(0x7fabf413d680 legacy=0x7fabf4148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf259c700 1 -- 172.21.15.143:0/2186041269 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf406e400 con 0x7fabf40a1170 2023-09-29T23:29:42.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf259c700 1 --2- 172.21.15.143:0/2186041269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf40a1170 0x7fabf413d140 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009e60 tx=0x7fabe4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8c0e7ffde7e7ed3 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:42.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf1d9b700 1 -- 172.21.15.143:0/2186041269 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe400e070 con 0x7fabf40a1170 2023-09-29T23:29:42.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf1d9b700 1 -- 172.21.15.143:0/2186041269 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe400bb60 con 0x7fabf40a1170 2023-09-29T23:29:42.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf1d9b700 1 -- 172.21.15.143:0/2186041269 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe4012430 con 0x7fabf40a1170 2023-09-29T23:29:42.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf892d700 1 -- 172.21.15.143:0/2186041269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf40a1170 msgr2=0x7fabf413d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf892d700 1 --2- 172.21.15.143:0/2186041269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf40a1170 0x7fabf413d140 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fabe4009e60 tx=0x7fabe4009780 comp rx=0 tx=0).stop 2023-09-29T23:29:42.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf892d700 1 -- 172.21.15.143:0/2186041269 shutdown_connections 2023-09-29T23:29:42.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf892d700 1 --2- 172.21.15.143:0/2186041269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf40a1170 0x7fabf413d140 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf892d700 1 -- 172.21.15.143:0/2186041269 >> 172.21.15.143:0/2186041269 conn(0x7fabf406f200 msgr2=0x7fabf4069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:42.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf892d700 1 -- 172.21.15.143:0/2186041269 shutdown_connections 2023-09-29T23:29:42.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf892d700 1 -- 172.21.15.143:0/2186041269 wait complete. 2023-09-29T23:29:42.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf892d700 1 Processor -- start 2023-09-29T23:29:42.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.376+0000 7fabf892d700 1 -- start start 2023-09-29T23:29:42.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf892d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fabf4066ec0 0x7fabf4098d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf892d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf40a1170 0x7fabf4140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf892d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabf413d680 0x7fabf41413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf892d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4141930 con 0x7fabf413d680 2023-09-29T23:29:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf892d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf41402d0 con 0x7fabf40a1170 2023-09-29T23:29:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf892d700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf4098920 con 0x7fabf4066ec0 2023-09-29T23:29:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf2d9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fabf4066ec0 0x7fabf4098d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf2d9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fabf4066ec0 0x7fabf4098d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60444/0 (socket says 172.21.15.143:60444) 2023-09-29T23:29:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf2d9d700 1 -- 172.21.15.143:0/3995935298 learned_addr learned my addr 172.21.15.143:0/3995935298 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf359e700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabf413d680 0x7fabf41413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf259c700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf40a1170 0x7fabf4140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf2d9d700 1 -- 172.21.15.143:0/3995935298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf40a1170 msgr2=0x7fabf4140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf2d9d700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf40a1170 0x7fabf4140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf2d9d700 1 -- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabf413d680 msgr2=0x7fabf41413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf2d9d700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabf413d680 0x7fabf41413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf2d9d700 1 -- 172.21.15.143:0/3995935298 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe4009a20 con 0x7fabf4066ec0 2023-09-29T23:29:42.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf2d9d700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fabf4066ec0 0x7fabf4098d40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fabe800c5c0 tx=0x7fabe800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:42.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabdaffd700 1 -- 172.21.15.143:0/3995935298 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8011590 con 0x7fabf4066ec0 2023-09-29T23:29:42.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabdaffd700 1 -- 172.21.15.143:0/3995935298 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe8011710 con 0x7fabf4066ec0 2023-09-29T23:29:42.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabdaffd700 1 -- 172.21.15.143:0/3995935298 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8011ad0 con 0x7fabf4066ec0 2023-09-29T23:29:42.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf892d700 1 -- 172.21.15.143:0/3995935298 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf41407a0 con 0x7fabf4066ec0 2023-09-29T23:29:42.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf892d700 1 -- 172.21.15.143:0/3995935298 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf4141b40 con 0x7fabf4066ec0 2023-09-29T23:29:42.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.380+0000 7fabf892d700 1 -- 172.21.15.143:0/3995935298 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf4141cd0 con 0x7fabf4066ec0 2023-09-29T23:29:42.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.384+0000 7fabdaffd700 1 -- 172.21.15.143:0/3995935298 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fabe801b3b0 con 0x7fabf4066ec0 2023-09-29T23:29:42.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.384+0000 7fabdaffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:42.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.384+0000 7fabdaffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:42.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.384+0000 7fabdaffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:42.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.384+0000 7fabdaffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:42.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.384+0000 7fabdaffd700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fabe008e8e0 0x7fabe0090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.384+0000 7fabf259c700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fabe008e8e0 0x7fabe0090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.384+0000 7fabdaffd700 1 -- 172.21.15.143:0/3995935298 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7fabe80b3d40 con 0x7fabf4066ec0 2023-09-29T23:29:42.388 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.384+0000 7fabf259c700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fabe008e8e0 0x7fabe0090d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fabe40097c0 tx=0x7fabe4019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:42.391 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.388+0000 7fabdaffd700 1 -- 172.21.15.143:0/3995935298 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fabe8084770 con 0x7fabf4066ec0 2023-09-29T23:29:42.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.544+0000 7fabf892d700 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2023-09-29T23:29:42.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.544+0000 7fabf892d700 1 -- 172.21.15.143:0/3995935298 --> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fabf4140b10 con 0x7fabe008e8e0 2023-09-29T23:29:42.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.560+0000 7fabdaffd700 1 -- 172.21.15.143:0/3995935298 <== mgr.4101 v2:172.21.15.143:6808/27602 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256577 (secure 0 0 0) 0x7fabf4140b10 con 0x7fabe008e8e0 2023-09-29T23:29:42.563 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.560+0000 7fabdaffd700 20 mgrc handle_command_reply tid 0 r 0 2023-09-29T23:29:42.567 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 10 mgrc shutdown 2023-09-29T23:29:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 -- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fabe008e8e0 msgr2=0x7fabe0090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fabe008e8e0 0x7fabe0090d90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fabe40097c0 tx=0x7fabe4019000 comp rx=0 tx=0).stop 2023-09-29T23:29:42.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 -- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fabf4066ec0 msgr2=0x7fabf4098d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fabf4066ec0 0x7fabf4098d40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fabe800c5c0 tx=0x7fabe800c8d0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 -- 172.21.15.143:0/3995935298 shutdown_connections 2023-09-29T23:29:42.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fabe008e8e0 0x7fabe0090d90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fabf413d680 0x7fabf41413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fabf40a1170 0x7fabf4140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 --2- 172.21.15.143:0/3995935298 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fabf4066ec0 0x7fabf4098d40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 -- 172.21.15.143:0/3995935298 >> 172.21.15.143:0/3995935298 conn(0x7fabf406f200 msgr2=0x7fabf40b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:42.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 -- 172.21.15.143:0/3995935298 shutdown_connections 2023-09-29T23:29:42.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.564+0000 7fabf892d700 1 -- 172.21.15.143:0/3995935298 wait complete. 2023-09-29T23:29:42.571 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T23:29:42.582 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-09-29T23:29:42.462750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":46032,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":45376,"kb_avail":377441328,"statfs":{"total":386547056640,"available":386499919872,"internally_reserved":0,"allocated":638976,"data_stored":1335992,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":46465024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.233117"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":162,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.772100+0000","last_change":"2023-09-29T23:29:29.623917+0000","last_active":"2023-09-29T23:29:35.772100+0000","last_peered":"2023-09-29T23:29:35.772100+0000","last_clean":"2023-09-29T23:29:35.772100+0000","last_became_active":"2023-09-29T23:29:29.621470+0000","last_became_peered":"2023-09-29T23:29:29.621470+0000","last_unstale":"2023-09-29T23:29:35.772100+0000","last_undegraded":"2023-09-29T23:29:35.772100+0000","last_fullsized":"2023-09-29T23:29:35.772100+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_clean_scrub_stamp":"2023-09-29T23:29:28.522101+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:37:09.181891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100857+0000","last_change":"2023-09-29T23:29:35.099334+0000","last_active":"2023-09-29T23:29:35.100857+0000","last_peered":"2023-09-29T23:29:35.100857+0000","last_clean":"2023-09-29T23:29:35.100857+0000","last_became_active":"2023-09-29T23:29:35.098819+0000","last_became_peered":"2023-09-29T23:29:35.098819+0000","last_unstale":"2023-09-29T23:29:35.100857+0000","last_undegraded":"2023-09-29T23:29:35.100857+0000","last_fullsized":"2023-09-29T23:29:35.100857+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:29:21.699082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100800+0000","last_change":"2023-09-29T23:29:35.098951+0000","last_active":"2023-09-29T23:29:35.100800+0000","last_peered":"2023-09-29T23:29:35.100800+0000","last_clean":"2023-09-29T23:29:35.100800+0000","last_became_active":"2023-09-29T23:29:35.098555+0000","last_became_peered":"2023-09-29T23:29:35.098555+0000","last_unstale":"2023-09-29T23:29:35.100800+0000","last_undegraded":"2023-09-29T23:29:35.100800+0000","last_fullsized":"2023-09-29T23:29:35.100800+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:43:14.246995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.095306+0000","last_change":"2023-09-29T23:29:35.095306+0000","last_active":"2023-09-29T23:29:35.095306+0000","last_peered":"2023-09-29T23:29:35.095306+0000","last_clean":"2023-09-29T23:29:35.095306+0000","last_became_active":"2023-09-29T23:29:35.093309+0000","last_became_peered":"2023-09-29T23:29:35.093309+0000","last_unstale":"2023-09-29T23:29:35.095306+0000","last_undegraded":"2023-09-29T23:29:35.095306+0000","last_fullsized":"2023-09-29T23:29:35.095306+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:10:21.305466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100907+0000","last_change":"2023-09-29T23:29:35.099643+0000","last_active":"2023-09-29T23:29:35.100907+0000","last_peered":"2023-09-29T23:29:35.100907+0000","last_clean":"2023-09-29T23:29:35.100907+0000","last_became_active":"2023-09-29T23:29:35.099061+0000","last_became_peered":"2023-09-29T23:29:35.099061+0000","last_unstale":"2023-09-29T23:29:35.100907+0000","last_undegraded":"2023-09-29T23:29:35.100907+0000","last_fullsized":"2023-09-29T23:29:35.100907+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:30.050339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.102979+0000","last_change":"2023-09-29T23:29:35.100113+0000","last_active":"2023-09-29T23:29:35.102979+0000","last_peered":"2023-09-29T23:29:35.102979+0000","last_clean":"2023-09-29T23:29:35.102979+0000","last_became_active":"2023-09-29T23:29:35.099578+0000","last_became_peered":"2023-09-29T23:29:35.099578+0000","last_unstale":"2023-09-29T23:29:35.102979+0000","last_undegraded":"2023-09-29T23:29:35.102979+0000","last_fullsized":"2023-09-29T23:29:35.102979+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:57:39.508565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100822+0000","last_change":"2023-09-29T23:29:35.099325+0000","last_active":"2023-09-29T23:29:35.100822+0000","last_peered":"2023-09-29T23:29:35.100822+0000","last_clean":"2023-09-29T23:29:35.100822+0000","last_became_active":"2023-09-29T23:29:35.098701+0000","last_became_peered":"2023-09-29T23:29:35.098701+0000","last_unstale":"2023-09-29T23:29:35.100822+0000","last_undegraded":"2023-09-29T23:29:35.100822+0000","last_fullsized":"2023-09-29T23:29:35.100822+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:35:07.769754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.098840+0000","last_change":"2023-09-29T23:29:35.096994+0000","last_active":"2023-09-29T23:29:35.098840+0000","last_peered":"2023-09-29T23:29:35.098840+0000","last_clean":"2023-09-29T23:29:35.098840+0000","last_became_active":"2023-09-29T23:29:35.096568+0000","last_became_peered":"2023-09-29T23:29:35.096568+0000","last_unstale":"2023-09-29T23:29:35.098840+0000","last_undegraded":"2023-09-29T23:29:35.098840+0000","last_fullsized":"2023-09-29T23:29:35.098840+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:22:06.698301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.777899+0000","last_change":"2023-09-29T23:29:35.776424+0000","last_active":"2023-09-29T23:29:35.777899+0000","last_peered":"2023-09-29T23:29:35.777899+0000","last_clean":"2023-09-29T23:29:35.777899+0000","last_became_active":"2023-09-29T23:29:35.775625+0000","last_became_peered":"2023-09-29T23:29:35.775625+0000","last_unstale":"2023-09-29T23:29:35.777899+0000","last_undegraded":"2023-09-29T23:29:35.777899+0000","last_fullsized":"2023-09-29T23:29:35.777899+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:46.078517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12224,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94359616,"statfs":{"total":96636764160,"available":96624246784,"internally_reserved":0,"allocated":122880,"data_stored":36810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12800,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94359040,"statfs":{"total":96636764160,"available":96623656960,"internally_reserved":0,"allocated":188416,"data_stored":627178,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12808,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94359032,"statfs":{"total":96636764160,"available":96623648768,"internally_reserved":0,"allocated":196608,"data_stored":631186,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8200,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363640,"statfs":{"total":96636764160,"available":96628367360,"internally_reserved":0,"allocated":131072,"data_stored":40818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T23:29:42.588 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T23:29:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 Processor -- start 2023-09-29T23:29:42.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 -- start start 2023-09-29T23:29:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd7d00b34e0 0x7fd7d009fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d009ec70 con 0x7fd7d0141d70 2023-09-29T23:29:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d0141ab0 con 0x7fd7d00b38b0 2023-09-29T23:29:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d009ea30 con 0x7fd7d00b34e0 2023-09-29T23:29:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7ced9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd7d0141d70 0x7fd7d01489b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54890/0 (socket says 172.21.15.143:54890) 2023-09-29T23:29:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7ced9d700 1 -- 172.21.15.143:0/322488375 learned_addr learned my addr 172.21.15.143:0/322488375 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:42.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7ce59c700 1 --2- 172.21.15.143:0/322488375 >> v2:172.21.15.156:3300/0 conn(0x7fd7d00b34e0 0x7fd7d009fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cdd9b700 1 -- 172.21.15.143:0/322488375 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288715976 0 0) 0x7fd7d009ec70 con 0x7fd7d0141d70 2023-09-29T23:29:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cdd9b700 1 -- 172.21.15.143:0/322488375 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7bc003570 con 0x7fd7d0141d70 2023-09-29T23:29:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cdd9b700 1 -- 172.21.15.143:0/322488375 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 486425735 0 0) 0x7fd7d0141ab0 con 0x7fd7d00b38b0 2023-09-29T23:29:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cdd9b700 1 -- 172.21.15.143:0/322488375 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7d009ec70 con 0x7fd7d00b38b0 2023-09-29T23:29:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7ce59c700 1 -- 172.21.15.143:0/322488375 >> v1:172.21.15.156:6789/0 conn(0x7fd7d00b38b0 legacy=0x7fd7d00a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7ce59c700 1 -- 172.21.15.143:0/322488375 >> v1:172.21.15.143:6789/0 conn(0x7fd7d0141d70 legacy=0x7fd7d01489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7ce59c700 1 -- 172.21.15.143:0/322488375 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d01347c0 con 0x7fd7d00b34e0 2023-09-29T23:29:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7ce59c700 1 --2- 172.21.15.143:0/322488375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d00b34e0 0x7fd7d009fcb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00098e0 tx=0x7fd7c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a26c69da6d83dacf server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cdd9b700 1 -- 172.21.15.143:0/322488375 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0011630 con 0x7fd7d00b34e0 2023-09-29T23:29:42.697 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cdd9b700 1 -- 172.21.15.143:0/322488375 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c0011790 con 0x7fd7d00b34e0 2023-09-29T23:29:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cdd9b700 1 -- 172.21.15.143:0/322488375 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0011b30 con 0x7fd7d00b34e0 2023-09-29T23:29:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 -- 172.21.15.143:0/322488375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d00b34e0 msgr2=0x7fd7d009fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 --2- 172.21.15.143:0/322488375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d00b34e0 0x7fd7d009fcb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00098e0 tx=0x7fd7c00091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 -- 172.21.15.143:0/322488375 shutdown_connections 2023-09-29T23:29:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 --2- 172.21.15.143:0/322488375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d00b34e0 0x7fd7d009fcb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 -- 172.21.15.143:0/322488375 >> 172.21.15.143:0/322488375 conn(0x7fd7d0062ed0 msgr2=0x7fd7d00632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 -- 172.21.15.143:0/322488375 shutdown_connections 2023-09-29T23:29:42.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.692+0000 7fd7cf59e700 1 -- 172.21.15.143:0/322488375 wait complete. 2023-09-29T23:29:42.700 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7cf59e700 1 Processor -- start 2023-09-29T23:29:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7cf59e700 1 -- start start 2023-09-29T23:29:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7cf59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d00b34e0 0x7fd7d00986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7cf59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00b38b0 0x7fd7d00997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7cf59e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d0141d70 0x7fd7d014e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7cf59e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00992e0 con 0x7fd7d00b38b0 2023-09-29T23:29:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7cf59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d009a110 con 0x7fd7d00b34e0 2023-09-29T23:29:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7cf59e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d014df10 con 0x7fd7d0141d70 2023-09-29T23:29:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c5d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00b38b0 0x7fd7d00997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c5d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00b38b0 0x7fd7d00997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47964/0 (socket says 172.21.15.143:47964) 2023-09-29T23:29:42.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c5d9b700 1 -- 172.21.15.143:0/347780396 learned_addr learned my addr 172.21.15.143:0/347780396 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7ced9d700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d0141d70 0x7fd7d014e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7ce59c700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d00b34e0 0x7fd7d00986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c5d9b700 1 -- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d0141d70 msgr2=0x7fd7d014e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c5d9b700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d0141d70 0x7fd7d014e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c5d9b700 1 -- 172.21.15.143:0/347780396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d00b34e0 msgr2=0x7fd7d00986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c5d9b700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d00b34e0 0x7fd7d00986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c5d9b700 1 -- 172.21.15.143:0/347780396 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c0009420 con 0x7fd7d00b38b0 2023-09-29T23:29:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c5d9b700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00b38b0 0x7fd7d00997e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800c790 tx=0x7fd7b8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c6ffd700 1 -- 172.21.15.143:0/347780396 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8010510 con 0x7fd7d00b38b0 2023-09-29T23:29:42.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c6ffd700 1 -- 172.21.15.143:0/347780396 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b8010670 con 0x7fd7d00b38b0 2023-09-29T23:29:42.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7cf59e700 1 -- 172.21.15.143:0/347780396 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d0099d20 con 0x7fd7d00b38b0 2023-09-29T23:29:42.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7c6ffd700 1 -- 172.21.15.143:0/347780396 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8010a10 con 0x7fd7d00b38b0 2023-09-29T23:29:42.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.696+0000 7fd7cf59e700 1 -- 172.21.15.143:0/347780396 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d013eeb0 con 0x7fd7d00b38b0 2023-09-29T23:29:42.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.700+0000 7fd7cf59e700 1 -- 172.21.15.143:0/347780396 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d005ecb0 con 0x7fd7d00b38b0 2023-09-29T23:29:42.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.700+0000 7fd7c6ffd700 1 -- 172.21.15.143:0/347780396 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fd7b801b3b0 con 0x7fd7d00b38b0 2023-09-29T23:29:42.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.700+0000 7fd7c6ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:42.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.704+0000 7fd7c6ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:42.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.704+0000 7fd7c6ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:42.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.704+0000 7fd7c6ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:42.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.704+0000 7fd7c6ffd700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7b008e8e0 0x7fd7b0090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.704+0000 7fd7ce59c700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7b008e8e0 0x7fd7b0090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.704+0000 7fd7c6ffd700 1 -- 172.21.15.143:0/347780396 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7fd7b80b4530 con 0x7fd7d00b38b0 2023-09-29T23:29:42.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.704+0000 7fd7ce59c700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7b008e8e0 0x7fd7b0090d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00093f0 tx=0x7fd7c0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:42.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.704+0000 7fd7c6ffd700 1 -- 172.21.15.143:0/347780396 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd7b8084ec0 con 0x7fd7d00b38b0 2023-09-29T23:29:42.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.860+0000 7fd7cf59e700 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2023-09-29T23:29:42.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.860+0000 7fd7cf59e700 1 -- 172.21.15.143:0/347780396 --> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7d00a0d40 con 0x7fd7b008e8e0 2023-09-29T23:29:42.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.876+0000 7fd7c6ffd700 1 -- 172.21.15.143:0/347780396 <== mgr.4101 v2:172.21.15.143:6808/27602 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256577 (secure 0 0 0) 0x7fd7d00a0d40 con 0x7fd7b008e8e0 2023-09-29T23:29:42.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.876+0000 7fd7c6ffd700 20 mgrc handle_command_reply tid 0 r 0 2023-09-29T23:29:42.882 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 10 mgrc shutdown 2023-09-29T23:29:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 -- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7b008e8e0 msgr2=0x7fd7b0090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7b008e8e0 0x7fd7b0090d90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00093f0 tx=0x7fd7c0017000 comp rx=0 tx=0).stop 2023-09-29T23:29:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 -- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00b38b0 msgr2=0x7fd7d00997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00b38b0 0x7fd7d00997e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800c790 tx=0x7fd7b8007e10 comp rx=0 tx=0).stop 2023-09-29T23:29:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 -- 172.21.15.143:0/347780396 shutdown_connections 2023-09-29T23:29:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7b008e8e0 0x7fd7b0090d90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d0141d70 0x7fd7d014e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00b38b0 0x7fd7d00997e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 --2- 172.21.15.143:0/347780396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d00b34e0 0x7fd7d00986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:42.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 -- 172.21.15.143:0/347780396 >> 172.21.15.143:0/347780396 conn(0x7fd7d0062ed0 msgr2=0x7fd7d00684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:42.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 -- 172.21.15.143:0/347780396 shutdown_connections 2023-09-29T23:29:42.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:42.880+0000 7fd7cf59e700 1 -- 172.21.15.143:0/347780396 wait complete. 2023-09-29T23:29:42.886 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T23:29:42.897 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-09-29T23:29:42.462750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":46032,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":45376,"kb_avail":377441328,"statfs":{"total":386547056640,"available":386499919872,"internally_reserved":0,"allocated":638976,"data_stored":1335992,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":46465024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.233117"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":162,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.772100+0000","last_change":"2023-09-29T23:29:29.623917+0000","last_active":"2023-09-29T23:29:35.772100+0000","last_peered":"2023-09-29T23:29:35.772100+0000","last_clean":"2023-09-29T23:29:35.772100+0000","last_became_active":"2023-09-29T23:29:29.621470+0000","last_became_peered":"2023-09-29T23:29:29.621470+0000","last_unstale":"2023-09-29T23:29:35.772100+0000","last_undegraded":"2023-09-29T23:29:35.772100+0000","last_fullsized":"2023-09-29T23:29:35.772100+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_clean_scrub_stamp":"2023-09-29T23:29:28.522101+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:37:09.181891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100857+0000","last_change":"2023-09-29T23:29:35.099334+0000","last_active":"2023-09-29T23:29:35.100857+0000","last_peered":"2023-09-29T23:29:35.100857+0000","last_clean":"2023-09-29T23:29:35.100857+0000","last_became_active":"2023-09-29T23:29:35.098819+0000","last_became_peered":"2023-09-29T23:29:35.098819+0000","last_unstale":"2023-09-29T23:29:35.100857+0000","last_undegraded":"2023-09-29T23:29:35.100857+0000","last_fullsized":"2023-09-29T23:29:35.100857+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:29:21.699082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100800+0000","last_change":"2023-09-29T23:29:35.098951+0000","last_active":"2023-09-29T23:29:35.100800+0000","last_peered":"2023-09-29T23:29:35.100800+0000","last_clean":"2023-09-29T23:29:35.100800+0000","last_became_active":"2023-09-29T23:29:35.098555+0000","last_became_peered":"2023-09-29T23:29:35.098555+0000","last_unstale":"2023-09-29T23:29:35.100800+0000","last_undegraded":"2023-09-29T23:29:35.100800+0000","last_fullsized":"2023-09-29T23:29:35.100800+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:43:14.246995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.095306+0000","last_change":"2023-09-29T23:29:35.095306+0000","last_active":"2023-09-29T23:29:35.095306+0000","last_peered":"2023-09-29T23:29:35.095306+0000","last_clean":"2023-09-29T23:29:35.095306+0000","last_became_active":"2023-09-29T23:29:35.093309+0000","last_became_peered":"2023-09-29T23:29:35.093309+0000","last_unstale":"2023-09-29T23:29:35.095306+0000","last_undegraded":"2023-09-29T23:29:35.095306+0000","last_fullsized":"2023-09-29T23:29:35.095306+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:10:21.305466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100907+0000","last_change":"2023-09-29T23:29:35.099643+0000","last_active":"2023-09-29T23:29:35.100907+0000","last_peered":"2023-09-29T23:29:35.100907+0000","last_clean":"2023-09-29T23:29:35.100907+0000","last_became_active":"2023-09-29T23:29:35.099061+0000","last_became_peered":"2023-09-29T23:29:35.099061+0000","last_unstale":"2023-09-29T23:29:35.100907+0000","last_undegraded":"2023-09-29T23:29:35.100907+0000","last_fullsized":"2023-09-29T23:29:35.100907+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:30.050339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.102979+0000","last_change":"2023-09-29T23:29:35.100113+0000","last_active":"2023-09-29T23:29:35.102979+0000","last_peered":"2023-09-29T23:29:35.102979+0000","last_clean":"2023-09-29T23:29:35.102979+0000","last_became_active":"2023-09-29T23:29:35.099578+0000","last_became_peered":"2023-09-29T23:29:35.099578+0000","last_unstale":"2023-09-29T23:29:35.102979+0000","last_undegraded":"2023-09-29T23:29:35.102979+0000","last_fullsized":"2023-09-29T23:29:35.102979+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:57:39.508565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100822+0000","last_change":"2023-09-29T23:29:35.099325+0000","last_active":"2023-09-29T23:29:35.100822+0000","last_peered":"2023-09-29T23:29:35.100822+0000","last_clean":"2023-09-29T23:29:35.100822+0000","last_became_active":"2023-09-29T23:29:35.098701+0000","last_became_peered":"2023-09-29T23:29:35.098701+0000","last_unstale":"2023-09-29T23:29:35.100822+0000","last_undegraded":"2023-09-29T23:29:35.100822+0000","last_fullsized":"2023-09-29T23:29:35.100822+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:35:07.769754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.098840+0000","last_change":"2023-09-29T23:29:35.096994+0000","last_active":"2023-09-29T23:29:35.098840+0000","last_peered":"2023-09-29T23:29:35.098840+0000","last_clean":"2023-09-29T23:29:35.098840+0000","last_became_active":"2023-09-29T23:29:35.096568+0000","last_became_peered":"2023-09-29T23:29:35.096568+0000","last_unstale":"2023-09-29T23:29:35.098840+0000","last_undegraded":"2023-09-29T23:29:35.098840+0000","last_fullsized":"2023-09-29T23:29:35.098840+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:22:06.698301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.777899+0000","last_change":"2023-09-29T23:29:35.776424+0000","last_active":"2023-09-29T23:29:35.777899+0000","last_peered":"2023-09-29T23:29:35.777899+0000","last_clean":"2023-09-29T23:29:35.777899+0000","last_became_active":"2023-09-29T23:29:35.775625+0000","last_became_peered":"2023-09-29T23:29:35.775625+0000","last_unstale":"2023-09-29T23:29:35.777899+0000","last_undegraded":"2023-09-29T23:29:35.777899+0000","last_fullsized":"2023-09-29T23:29:35.777899+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:46.078517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12224,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94359616,"statfs":{"total":96636764160,"available":96624246784,"internally_reserved":0,"allocated":122880,"data_stored":36810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12800,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94359040,"statfs":{"total":96636764160,"available":96623656960,"internally_reserved":0,"allocated":188416,"data_stored":627178,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12808,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94359032,"statfs":{"total":96636764160,"available":96623648768,"internally_reserved":0,"allocated":196608,"data_stored":631186,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8200,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363640,"statfs":{"total":96636764160,"available":96628367360,"internally_reserved":0,"allocated":131072,"data_stored":40818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T23:29:42.902 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T23:29:43.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.008+0000 7f65a7375700 1 Processor -- start 2023-09-29T23:29:43.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.008+0000 7f65a7375700 1 -- start start 2023-09-29T23:29:43.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.008+0000 7f65a7375700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a0066ec0 0x7f65a009fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.008+0000 7f65a7375700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f65a006b680 0x7f65a00a0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.008+0000 7f65a7375700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f65a009f450 0x7f65a0135e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.008+0000 7f65a7375700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a0067290 con 0x7f65a006b680 2023-09-29T23:29:43.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.008+0000 7f65a7375700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65a00a0b40 con 0x7f65a009f450 2023-09-29T23:29:43.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.008+0000 7f65a7375700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a0067ad0 con 0x7f65a0066ec0 2023-09-29T23:29:43.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5b72700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f65a006b680 0x7f65a00a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5b72700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f65a006b680 0x7f65a00a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34398/0 (socket says 172.21.15.143:34398) 2023-09-29T23:29:43.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5b72700 1 -- 172.21.15.143:0/915219353 learned_addr learned my addr 172.21.15.143:0/915219353 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:43.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a6373700 1 --2- 172.21.15.143:0/915219353 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a0066ec0 0x7f65a009fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a6b74700 1 --2- 172.21.15.143:0/915219353 >> v2:172.21.15.156:3300/0 conn(0x7f65a009f450 0x7f65a0135e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5b72700 1 -- 172.21.15.143:0/915219353 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a0066ec0 msgr2=0x7f65a009fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:43.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5b72700 1 --2- 172.21.15.143:0/915219353 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a0066ec0 0x7f65a009fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5b72700 1 -- 172.21.15.143:0/915219353 >> v2:172.21.15.156:3300/0 conn(0x7f65a009f450 msgr2=0x7f65a0135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:43.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5b72700 1 --2- 172.21.15.143:0/915219353 >> v2:172.21.15.156:3300/0 conn(0x7f65a009f450 0x7f65a0135e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5b72700 1 -- 172.21.15.143:0/915219353 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a00a0dc0 con 0x7f65a006b680 2023-09-29T23:29:43.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a6b74700 1 --2- 172.21.15.143:0/915219353 >> v2:172.21.15.156:3300/0 conn(0x7f65a009f450 0x7f65a0135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:43.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5b72700 1 --2- 172.21.15.143:0/915219353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65a006b680 0x7f65a00a0190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65a00a0fb0 tx=0x7f659c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e408bd90638b37aa server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:43.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5371700 1 -- 172.21.15.143:0/915219353 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00e070 con 0x7f65a006b680 2023-09-29T23:29:43.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5371700 1 -- 172.21.15.143:0/915219353 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c00b9f0 con 0x7f65a006b680 2023-09-29T23:29:43.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a5371700 1 -- 172.21.15.143:0/915219353 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00bdb0 con 0x7f65a006b680 2023-09-29T23:29:43.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 -- 172.21.15.143:0/915219353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65a006b680 msgr2=0x7f65a00a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 --2- 172.21.15.143:0/915219353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65a006b680 0x7f65a00a0190 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f65a00a0fb0 tx=0x7f659c009840 comp rx=0 tx=0).stop 2023-09-29T23:29:43.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 -- 172.21.15.143:0/915219353 shutdown_connections 2023-09-29T23:29:43.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 --2- 172.21.15.143:0/915219353 >> v2:172.21.15.156:3300/0 conn(0x7f65a009f450 0x7f65a0135e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 --2- 172.21.15.143:0/915219353 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65a006b680 0x7f65a00a0190 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 --2- 172.21.15.143:0/915219353 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a0066ec0 0x7f65a009fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 -- 172.21.15.143:0/915219353 >> 172.21.15.143:0/915219353 conn(0x7f65a006f200 msgr2=0x7f65a0069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:43.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 -- 172.21.15.143:0/915219353 shutdown_connections 2023-09-29T23:29:43.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 -- 172.21.15.143:0/915219353 wait complete. 2023-09-29T23:29:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 Processor -- start 2023-09-29T23:29:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 -- start start 2023-09-29T23:29:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65a0066ec0 0x7f65a009d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65a006b680 0x7f65a00b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.012+0000 7f65a7375700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a009f450 0x7f65a01532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a7375700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a014fb50 con 0x7f65a0066ec0 2023-09-29T23:29:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a7375700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65a006e320 con 0x7f65a006b680 2023-09-29T23:29:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a7375700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65a00b1ea0 con 0x7f65a009f450 2023-09-29T23:29:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a6b74700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a009f450 0x7f65a01532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a6373700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65a0066ec0 0x7f65a009d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a6b74700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a009f450 0x7f65a01532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60314/0 (socket says 172.21.15.143:60314) 2023-09-29T23:29:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a6b74700 1 -- 172.21.15.143:0/1698737537 learned_addr learned my addr 172.21.15.143:0/1698737537 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a5b72700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65a006b680 0x7f65a00b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a6b74700 1 -- 172.21.15.143:0/1698737537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65a006b680 msgr2=0x7f65a00b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a6b74700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65a006b680 0x7f65a00b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a6b74700 1 -- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65a0066ec0 msgr2=0x7f65a009d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a6b74700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65a0066ec0 0x7f65a009d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a6b74700 1 -- 172.21.15.143:0/1698737537 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c009ae0 con 0x7f65a009f450 2023-09-29T23:29:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a6b74700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a009f450 0x7f65a01532a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f659800e870 tx=0x7f6598008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65967fc700 1 -- 172.21.15.143:0/1698737537 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6598013410 con 0x7f65a009f450 2023-09-29T23:29:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65967fc700 1 -- 172.21.15.143:0/1698737537 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6598013570 con 0x7f65a009f450 2023-09-29T23:29:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a7375700 1 -- 172.21.15.143:0/1698737537 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65a009ed10 con 0x7f65a009f450 2023-09-29T23:29:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65967fc700 1 -- 172.21.15.143:0/1698737537 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65980138f0 con 0x7f65a009f450 2023-09-29T23:29:43.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a7375700 1 -- 172.21.15.143:0/1698737537 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65a009b140 con 0x7f65a009f450 2023-09-29T23:29:43.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.016+0000 7f65a7375700 1 -- 172.21.15.143:0/1698737537 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65a005ecb0 con 0x7f65a009f450 2023-09-29T23:29:43.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.020+0000 7f65967fc700 1 -- 172.21.15.143:0/1698737537 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f6598013ca0 con 0x7f65a009f450 2023-09-29T23:29:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.020+0000 7f65967fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.020+0000 7f65967fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.020+0000 7f65967fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.020+0000 7f65967fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.020+0000 7f65967fc700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f658408e8e0 0x7f6584090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.020+0000 7f65a6373700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f658408e8e0 0x7f6584090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.020+0000 7f65967fc700 1 -- 172.21.15.143:0/1698737537 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f65980b49a0 con 0x7f65a009f450 2023-09-29T23:29:43.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.020+0000 7f65a6373700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f658408e8e0 0x7f6584090d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6590009d10 tx=0x7f65900091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:43.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.024+0000 7f65967fc700 1 -- 172.21.15.143:0/1698737537 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6598088dc0 con 0x7f65a009f450 2023-09-29T23:29:43.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.180+0000 7f65a7375700 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2023-09-29T23:29:43.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.180+0000 7f65a7375700 1 -- 172.21.15.143:0/1698737537 --> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65a006e460 con 0x7f658408e8e0 2023-09-29T23:29:43.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.192+0000 7f65967fc700 1 -- 172.21.15.143:0/1698737537 <== mgr.4101 v2:172.21.15.143:6808/27602 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256577 (secure 0 0 0) 0x7f65a006e460 con 0x7f658408e8e0 2023-09-29T23:29:43.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.192+0000 7f65967fc700 20 mgrc handle_command_reply tid 0 r 0 2023-09-29T23:29:43.199 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:43.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 10 mgrc shutdown 2023-09-29T23:29:43.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 -- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f658408e8e0 msgr2=0x7f6584090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f658408e8e0 0x7f6584090d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6590009d10 tx=0x7f65900091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 -- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a009f450 msgr2=0x7f65a01532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a009f450 0x7f65a01532a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f659800e870 tx=0x7f6598008040 comp rx=0 tx=0).stop 2023-09-29T23:29:43.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 -- 172.21.15.143:0/1698737537 shutdown_connections 2023-09-29T23:29:43.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f658408e8e0 0x7f6584090d90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f65a009f450 0x7f65a01532a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65a006b680 0x7f65a00b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 --2- 172.21.15.143:0/1698737537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65a0066ec0 0x7f65a009d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 -- 172.21.15.143:0/1698737537 >> 172.21.15.143:0/1698737537 conn(0x7f65a006f200 msgr2=0x7f65a01352e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:43.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 -- 172.21.15.143:0/1698737537 shutdown_connections 2023-09-29T23:29:43.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.196+0000 7f65a7375700 1 -- 172.21.15.143:0/1698737537 wait complete. 2023-09-29T23:29:43.202 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T23:29:43.214 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-09-29T23:29:42.462750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":46032,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":45376,"kb_avail":377441328,"statfs":{"total":386547056640,"available":386499919872,"internally_reserved":0,"allocated":638976,"data_stored":1335992,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":46465024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.233117"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":162,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.772100+0000","last_change":"2023-09-29T23:29:29.623917+0000","last_active":"2023-09-29T23:29:35.772100+0000","last_peered":"2023-09-29T23:29:35.772100+0000","last_clean":"2023-09-29T23:29:35.772100+0000","last_became_active":"2023-09-29T23:29:29.621470+0000","last_became_peered":"2023-09-29T23:29:29.621470+0000","last_unstale":"2023-09-29T23:29:35.772100+0000","last_undegraded":"2023-09-29T23:29:35.772100+0000","last_fullsized":"2023-09-29T23:29:35.772100+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_clean_scrub_stamp":"2023-09-29T23:29:28.522101+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:37:09.181891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100857+0000","last_change":"2023-09-29T23:29:35.099334+0000","last_active":"2023-09-29T23:29:35.100857+0000","last_peered":"2023-09-29T23:29:35.100857+0000","last_clean":"2023-09-29T23:29:35.100857+0000","last_became_active":"2023-09-29T23:29:35.098819+0000","last_became_peered":"2023-09-29T23:29:35.098819+0000","last_unstale":"2023-09-29T23:29:35.100857+0000","last_undegraded":"2023-09-29T23:29:35.100857+0000","last_fullsized":"2023-09-29T23:29:35.100857+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:29:21.699082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100800+0000","last_change":"2023-09-29T23:29:35.098951+0000","last_active":"2023-09-29T23:29:35.100800+0000","last_peered":"2023-09-29T23:29:35.100800+0000","last_clean":"2023-09-29T23:29:35.100800+0000","last_became_active":"2023-09-29T23:29:35.098555+0000","last_became_peered":"2023-09-29T23:29:35.098555+0000","last_unstale":"2023-09-29T23:29:35.100800+0000","last_undegraded":"2023-09-29T23:29:35.100800+0000","last_fullsized":"2023-09-29T23:29:35.100800+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:43:14.246995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.095306+0000","last_change":"2023-09-29T23:29:35.095306+0000","last_active":"2023-09-29T23:29:35.095306+0000","last_peered":"2023-09-29T23:29:35.095306+0000","last_clean":"2023-09-29T23:29:35.095306+0000","last_became_active":"2023-09-29T23:29:35.093309+0000","last_became_peered":"2023-09-29T23:29:35.093309+0000","last_unstale":"2023-09-29T23:29:35.095306+0000","last_undegraded":"2023-09-29T23:29:35.095306+0000","last_fullsized":"2023-09-29T23:29:35.095306+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:10:21.305466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100907+0000","last_change":"2023-09-29T23:29:35.099643+0000","last_active":"2023-09-29T23:29:35.100907+0000","last_peered":"2023-09-29T23:29:35.100907+0000","last_clean":"2023-09-29T23:29:35.100907+0000","last_became_active":"2023-09-29T23:29:35.099061+0000","last_became_peered":"2023-09-29T23:29:35.099061+0000","last_unstale":"2023-09-29T23:29:35.100907+0000","last_undegraded":"2023-09-29T23:29:35.100907+0000","last_fullsized":"2023-09-29T23:29:35.100907+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:30.050339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.102979+0000","last_change":"2023-09-29T23:29:35.100113+0000","last_active":"2023-09-29T23:29:35.102979+0000","last_peered":"2023-09-29T23:29:35.102979+0000","last_clean":"2023-09-29T23:29:35.102979+0000","last_became_active":"2023-09-29T23:29:35.099578+0000","last_became_peered":"2023-09-29T23:29:35.099578+0000","last_unstale":"2023-09-29T23:29:35.102979+0000","last_undegraded":"2023-09-29T23:29:35.102979+0000","last_fullsized":"2023-09-29T23:29:35.102979+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:57:39.508565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100822+0000","last_change":"2023-09-29T23:29:35.099325+0000","last_active":"2023-09-29T23:29:35.100822+0000","last_peered":"2023-09-29T23:29:35.100822+0000","last_clean":"2023-09-29T23:29:35.100822+0000","last_became_active":"2023-09-29T23:29:35.098701+0000","last_became_peered":"2023-09-29T23:29:35.098701+0000","last_unstale":"2023-09-29T23:29:35.100822+0000","last_undegraded":"2023-09-29T23:29:35.100822+0000","last_fullsized":"2023-09-29T23:29:35.100822+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:35:07.769754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.098840+0000","last_change":"2023-09-29T23:29:35.096994+0000","last_active":"2023-09-29T23:29:35.098840+0000","last_peered":"2023-09-29T23:29:35.098840+0000","last_clean":"2023-09-29T23:29:35.098840+0000","last_became_active":"2023-09-29T23:29:35.096568+0000","last_became_peered":"2023-09-29T23:29:35.096568+0000","last_unstale":"2023-09-29T23:29:35.098840+0000","last_undegraded":"2023-09-29T23:29:35.098840+0000","last_fullsized":"2023-09-29T23:29:35.098840+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:22:06.698301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.777899+0000","last_change":"2023-09-29T23:29:35.776424+0000","last_active":"2023-09-29T23:29:35.777899+0000","last_peered":"2023-09-29T23:29:35.777899+0000","last_clean":"2023-09-29T23:29:35.777899+0000","last_became_active":"2023-09-29T23:29:35.775625+0000","last_became_peered":"2023-09-29T23:29:35.775625+0000","last_unstale":"2023-09-29T23:29:35.777899+0000","last_undegraded":"2023-09-29T23:29:35.777899+0000","last_fullsized":"2023-09-29T23:29:35.777899+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:46.078517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12224,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94359616,"statfs":{"total":96636764160,"available":96624246784,"internally_reserved":0,"allocated":122880,"data_stored":36810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12800,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94359040,"statfs":{"total":96636764160,"available":96623656960,"internally_reserved":0,"allocated":188416,"data_stored":627178,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12808,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94359032,"statfs":{"total":96636764160,"available":96623648768,"internally_reserved":0,"allocated":196608,"data_stored":631186,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8200,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363640,"statfs":{"total":96636764160,"available":96628367360,"internally_reserved":0,"allocated":131072,"data_stored":40818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T23:29:43.219 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3f is not active+clean 2023-09-29T23:29:43.219 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3e is not active+clean 2023-09-29T23:29:43.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3d is not active+clean 2023-09-29T23:29:43.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3c is not active+clean 2023-09-29T23:29:43.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.220 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3b is not active+clean 2023-09-29T23:29:43.220 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3a is not active+clean 2023-09-29T23:29:43.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.39 is not active+clean 2023-09-29T23:29:43.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.38 is not active+clean 2023-09-29T23:29:43.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.37 is not active+clean 2023-09-29T23:29:43.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.221 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.36 is not active+clean 2023-09-29T23:29:43.221 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.35 is not active+clean 2023-09-29T23:29:43.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.34 is not active+clean 2023-09-29T23:29:43.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.33 is not active+clean 2023-09-29T23:29:43.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.32 is not active+clean 2023-09-29T23:29:43.222 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.222 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.31 is not active+clean 2023-09-29T23:29:43.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.30 is not active+clean 2023-09-29T23:29:43.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2f is not active+clean 2023-09-29T23:29:43.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2e is not active+clean 2023-09-29T23:29:43.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.223 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2d is not active+clean 2023-09-29T23:29:43.223 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2c is not active+clean 2023-09-29T23:29:43.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2b is not active+clean 2023-09-29T23:29:43.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2a is not active+clean 2023-09-29T23:29:43.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.29 is not active+clean 2023-09-29T23:29:43.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.224 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.28 is not active+clean 2023-09-29T23:29:43.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.27 is not active+clean 2023-09-29T23:29:43.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.26 is not active+clean 2023-09-29T23:29:43.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.25 is not active+clean 2023-09-29T23:29:43.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.24 is not active+clean 2023-09-29T23:29:43.225 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.225 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.23 is not active+clean 2023-09-29T23:29:43.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.22 is not active+clean 2023-09-29T23:29:43.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.21 is not active+clean 2023-09-29T23:29:43.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.20 is not active+clean 2023-09-29T23:29:43.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-09-29T23:29:43.226 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.226 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-09-29T23:29:43.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-09-29T23:29:43.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2023-09-29T23:29:43.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-09-29T23:29:43.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-09-29T23:29:43.227 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.227 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-09-29T23:29:43.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-09-29T23:29:43.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-09-29T23:29:43.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-09-29T23:29:43.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-09-29T23:29:43.228 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.228 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-09-29T23:29:43.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-09-29T23:29:43.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-09-29T23:29:43.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-09-29T23:29:43.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.229 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-09-29T23:29:43.229 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-09-29T23:29:43.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-09-29T23:29:43.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-09-29T23:29:43.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-09-29T23:29:43.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.230 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2023-09-29T23:29:43.230 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-09-29T23:29:43.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-09-29T23:29:43.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-09-29T23:29:43.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-09-29T23:29:43.231 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.231 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-09-29T23:29:43.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2023-09-29T23:29:43.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-09-29T23:29:43.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2023-09-29T23:29:43.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.232 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-09-29T23:29:43.232 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-09-29T23:29:43.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-09-29T23:29:43.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-09-29T23:29:43.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-09-29T23:29:43.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.233 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2023-09-29T23:29:43.233 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-09-29T23:29:43.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-09-29T23:29:43.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-09-29T23:29:43.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2023-09-29T23:29:43.234 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.234 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-09-29T23:29:43.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-09-29T23:29:43.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2023-09-29T23:29:43.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-09-29T23:29:43.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-09-29T23:29:43.235 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.235 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-09-29T23:29:43.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-09-29T23:29:43.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2023-09-29T23:29:43.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-09-29T23:29:43.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-09-29T23:29:43.236 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.236 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-09-29T23:29:43.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-09-29T23:29:43.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2023-09-29T23:29:43.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-09-29T23:29:43.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.237 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2023-09-29T23:29:43.237 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-09-29T23:29:43.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2023-09-29T23:29:43.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-09-29T23:29:43.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2023-09-29T23:29:43.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.238 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-09-29T23:29:43.238 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2023-09-29T23:29:43.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-09-29T23:29:43.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-09-29T23:29:43.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-09-29T23:29:43.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:39.227089+0000', 'last_change': '2023-09-29T23:29:39.227089+0000', 'last_active': '2023-09-29T23:29:39.227089+0000', 'last_peered': '2023-09-29T23:29:39.227089+0000', 'last_clean': '2023-09-29T23:29:39.227089+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:39.227089+0000', 'last_undegraded': '2023-09-29T23:29:39.227089+0000', 'last_fullsized': '2023-09-29T23:29:39.227089+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:39.227089+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.239 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2023-09-29T23:29:43.239 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-09-29T23:29:38.223331+0000', 'last_change': '2023-09-29T23:29:38.223331+0000', 'last_active': '2023-09-29T23:29:38.223331+0000', 'last_peered': '2023-09-29T23:29:38.223331+0000', 'last_clean': '2023-09-29T23:29:38.223331+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-09-29T23:29:38.223331+0000', 'last_undegraded': '2023-09-29T23:29:38.223331+0000', 'last_fullsized': '2023-09-29T23:29:38.223331+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'last_clean_scrub_stamp': '2023-09-29T23:29:38.223331+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-09-29T23:29:43.240 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-29T23:29:43.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.348+0000 7fe63e92a700 1 Processor -- start 2023-09-29T23:29:43.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.348+0000 7fe63e92a700 1 -- start start 2023-09-29T23:29:43.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.348+0000 7fe63e92a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe6380a5f20 0x7fe63806baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.348+0000 7fe63e92a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe638134100 0x7fe638134540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.348+0000 7fe63e92a700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6381425b0 con 0x7fe638141660 2023-09-29T23:29:43.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.348+0000 7fe63e92a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6380a21a0 con 0x7fe638134100 2023-09-29T23:29:43.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.348+0000 7fe63e92a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6380a62f0 con 0x7fe6380a5f20 2023-09-29T23:29:43.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d127700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe638134100 0x7fe638134540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d127700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe638134100 0x7fe638134540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34412/0 (socket says 172.21.15.143:34412) 2023-09-29T23:29:43.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d127700 1 -- 172.21.15.143:0/138551334 learned_addr learned my addr 172.21.15.143:0/138551334 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:43.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e129700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe638141660 0x7fe638141a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60624/0 (socket says 172.21.15.143:60624) 2023-09-29T23:29:43.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d928700 1 --2- 172.21.15.143:0/138551334 >> v2:172.21.15.156:3300/0 conn(0x7fe6380a5f20 0x7fe63806baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d127700 1 -- 172.21.15.143:0/138551334 >> v2:172.21.15.156:3300/0 conn(0x7fe6380a5f20 msgr2=0x7fe63806baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:43.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d127700 1 --2- 172.21.15.143:0/138551334 >> v2:172.21.15.156:3300/0 conn(0x7fe6380a5f20 0x7fe63806baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d127700 1 -- 172.21.15.143:0/138551334 >> v1:172.21.15.143:6789/0 conn(0x7fe638141660 legacy=0x7fe638141a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d127700 1 -- 172.21.15.143:0/138551334 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6381422f0 con 0x7fe638134100 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63c926700 1 -- 172.21.15.143:0/138551334 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265398236 0 0) 0x7fe6381425b0 con 0x7fe638141660 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d127700 1 --2- 172.21.15.143:0/138551334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe638134100 0x7fe638134540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe638064070 tx=0x7fe628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feac40d7c3048da server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63c926700 1 -- 172.21.15.143:0/138551334 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe628009e70 con 0x7fe638134100 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63c926700 1 -- 172.21.15.143:0/138551334 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62800b8f0 con 0x7fe638134100 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63c926700 1 -- 172.21.15.143:0/138551334 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62800bc90 con 0x7fe638134100 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 -- 172.21.15.143:0/138551334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe638134100 msgr2=0x7fe638134540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 --2- 172.21.15.143:0/138551334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe638134100 0x7fe638134540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe638064070 tx=0x7fe628009250 comp rx=0 tx=0).stop 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 -- 172.21.15.143:0/138551334 shutdown_connections 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 --2- 172.21.15.143:0/138551334 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe638134100 0x7fe638134540 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 --2- 172.21.15.143:0/138551334 >> v2:172.21.15.156:3300/0 conn(0x7fe6380a5f20 0x7fe63806baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 -- 172.21.15.143:0/138551334 >> 172.21.15.143:0/138551334 conn(0x7fe638067da0 msgr2=0x7fe6380681a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:43.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 -- 172.21.15.143:0/138551334 shutdown_connections 2023-09-29T23:29:43.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 -- 172.21.15.143:0/138551334 wait complete. 2023-09-29T23:29:43.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 Processor -- start 2023-09-29T23:29:43.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 -- start start 2023-09-29T23:29:43.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6380a5f20 0x7fe638146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe638134100 0x7fe63806d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe638141660 0x7fe6381513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe638151d20 con 0x7fe638134100 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe638151fe0 con 0x7fe6380a5f20 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e92a700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe63806e280 con 0x7fe638141660 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d127700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe638134100 0x7fe63806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d127700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe638134100 0x7fe63806d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34428/0 (socket says 172.21.15.143:34428) 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63d127700 1 -- 172.21.15.143:0/3842164260 learned_addr learned my addr 172.21.15.143:0/3842164260 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.352+0000 7fe63e129700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe638141660 0x7fe6381513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe63d928700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6380a5f20 0x7fe638146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe63d127700 1 -- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe638141660 msgr2=0x7fe6381513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe63d127700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe638141660 0x7fe6381513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe63d127700 1 -- 172.21.15.143:0/3842164260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6380a5f20 msgr2=0x7fe638146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe63d127700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6380a5f20 0x7fe638146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe63d127700 1 -- 172.21.15.143:0/3842164260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6280094f0 con 0x7fe638134100 2023-09-29T23:29:43.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe63d127700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe638134100 0x7fe63806d600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6280094c0 tx=0x7fe628017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:43.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe62dffb700 1 -- 172.21.15.143:0/3842164260 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62800bc70 con 0x7fe638134100 2023-09-29T23:29:43.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe62dffb700 1 -- 172.21.15.143:0/3842164260 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe628024440 con 0x7fe638134100 2023-09-29T23:29:43.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe63e92a700 1 -- 172.21.15.143:0/3842164260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe638147870 con 0x7fe638134100 2023-09-29T23:29:43.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe62dffb700 1 -- 172.21.15.143:0/3842164260 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe628024800 con 0x7fe638134100 2023-09-29T23:29:43.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe63e92a700 1 -- 172.21.15.143:0/3842164260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe638147620 con 0x7fe638134100 2023-09-29T23:29:43.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.356+0000 7fe63e92a700 1 -- 172.21.15.143:0/3842164260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63805ecb0 con 0x7fe638134100 2023-09-29T23:29:43.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.360+0000 7fe62dffb700 1 -- 172.21.15.143:0/3842164260 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fe62801a420 con 0x7fe638134100 2023-09-29T23:29:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.360+0000 7fe62dffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:43.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.360+0000 7fe62dffb700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:43.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.360+0000 7fe62dffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:43.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.360+0000 7fe62dffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:43.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.360+0000 7fe62dffb700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe61c08e8b0 0x7fe61c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.360+0000 7fe62dffb700 1 -- 172.21.15.143:0/3842164260 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7fe6280b5df0 con 0x7fe638134100 2023-09-29T23:29:43.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.360+0000 7fe63d928700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe61c08e8b0 0x7fe61c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.360+0000 7fe63d928700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe61c08e8b0 0x7fe61c090d60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe634009db0 tx=0x7fe6340091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:43.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.368+0000 7fe62dffb700 1 -- 172.21.15.143:0/3842164260 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe628086810 con 0x7fe638134100 2023-09-29T23:29:43.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.588+0000 7fe63e92a700 1 -- 172.21.15.143:0/3842164260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe6380a1e10 con 0x7fe638134100 2023-09-29T23:29:43.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.588+0000 7fe62dffb700 1 -- 172.21.15.143:0/3842164260 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1203 (secure 0 0 0) 0x7fe62802a4d0 con 0x7fe638134100 2023-09-29T23:29:43.591 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:43.591 INFO:teuthology.orchestra.run.smithi143.stdout:{"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","health":{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":25,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":19,"num_osds":4,"num_up_osds":4,"osd_up_since":1696030167,"num_in_osds":4,"osd_in_since":1696030159,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":47136768,"bytes_avail":386499919872,"bytes_total":386547056640,"unknown_pgs_ratio":0.91428571939468384},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"c","status":"up:active","gid":4258}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-09-29T23:29:18.318969+0000","services":{}},"progress_events":{}} 2023-09-29T23:29:43.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 10 mgrc shutdown 2023-09-29T23:29:43.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 -- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe61c08e8b0 msgr2=0x7fe61c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe61c08e8b0 0x7fe61c090d60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe634009db0 tx=0x7fe6340091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 -- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe638134100 msgr2=0x7fe63806d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe638134100 0x7fe63806d600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6280094c0 tx=0x7fe628017000 comp rx=0 tx=0).stop 2023-09-29T23:29:43.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 -- 172.21.15.143:0/3842164260 shutdown_connections 2023-09-29T23:29:43.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe61c08e8b0 0x7fe61c090d60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe638141660 0x7fe6381513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe638134100 0x7fe63806d600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 --2- 172.21.15.143:0/3842164260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6380a5f20 0x7fe638146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 -- 172.21.15.143:0/3842164260 >> 172.21.15.143:0/3842164260 conn(0x7fe638067da0 msgr2=0x7fe638141a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:43.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 -- 172.21.15.143:0/3842164260 shutdown_connections 2023-09-29T23:29:43.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.592+0000 7fe63e92a700 1 -- 172.21.15.143:0/3842164260 wait complete. 2023-09-29T23:29:43.606 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-29T23:29:43.606 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T23:29:43.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb0949ed700 1 Processor -- start 2023-09-29T23:29:43.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb0949ed700 1 -- start start 2023-09-29T23:29:43.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb0949ed700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb0900a5310 0x7fb0900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb0949ed700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0900a48e0 con 0x7fb09013fce0 2023-09-29T23:29:43.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb0949ed700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0900b7600 con 0x7fb0900b71e0 2023-09-29T23:29:43.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb0949ed700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0900a3de0 con 0x7fb0900a5310 2023-09-29T23:29:43.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08f59e700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb09013fce0 0x7fb09014aa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60632/0 (socket says 172.21.15.143:60632) 2023-09-29T23:29:43.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08f59e700 1 -- 172.21.15.143:0/3503407113 learned_addr learned my addr 172.21.15.143:0/3503407113 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:43.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08ed9d700 1 --2- 172.21.15.143:0/3503407113 >> v2:172.21.15.156:3300/0 conn(0x7fb0900a5310 0x7fb0900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08dd9b700 1 -- 172.21.15.143:0/3503407113 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248707130 0 0) 0x7fb0900a48e0 con 0x7fb09013fce0 2023-09-29T23:29:43.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08dd9b700 1 -- 172.21.15.143:0/3503407113 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb070003570 con 0x7fb09013fce0 2023-09-29T23:29:43.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08dd9b700 1 -- 172.21.15.143:0/3503407113 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2938159966 0 0) 0x7fb070003570 con 0x7fb09013fce0 2023-09-29T23:29:43.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08ed9d700 1 -- 172.21.15.143:0/3503407113 >> v1:172.21.15.156:6789/0 conn(0x7fb0900b71e0 legacy=0x7fb09013f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08ed9d700 1 -- 172.21.15.143:0/3503407113 >> v1:172.21.15.143:6789/0 conn(0x7fb09013fce0 legacy=0x7fb09014aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08ed9d700 1 -- 172.21.15.143:0/3503407113 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0900a40a0 con 0x7fb0900a5310 2023-09-29T23:29:43.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08ed9d700 1 --2- 172.21.15.143:0/3503407113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0900a5310 0x7fb0900a3770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb0780098e0 tx=0x7fb0780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aa7f57ae72cac9d server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:43.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08dd9b700 1 -- 172.21.15.143:0/3503407113 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088001490 con 0x7fb0900a5310 2023-09-29T23:29:43.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08dd9b700 1 -- 172.21.15.143:0/3503407113 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb078011420 con 0x7fb0900a5310 2023-09-29T23:29:43.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.708+0000 7fb08dd9b700 1 -- 172.21.15.143:0/3503407113 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0780117a0 con 0x7fb0900a5310 2023-09-29T23:29:43.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 -- 172.21.15.143:0/3503407113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0900a5310 msgr2=0x7fb0900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 --2- 172.21.15.143:0/3503407113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0900a5310 0x7fb0900a3770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb0780098e0 tx=0x7fb0780091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 -- 172.21.15.143:0/3503407113 shutdown_connections 2023-09-29T23:29:43.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 --2- 172.21.15.143:0/3503407113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0900a5310 0x7fb0900a3770 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 -- 172.21.15.143:0/3503407113 >> 172.21.15.143:0/3503407113 conn(0x7fb090066c90 msgr2=0x7fb090067090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:43.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 -- 172.21.15.143:0/3503407113 shutdown_connections 2023-09-29T23:29:43.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 -- 172.21.15.143:0/3503407113 wait complete. 2023-09-29T23:29:43.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 Processor -- start 2023-09-29T23:29:43.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 -- start start 2023-09-29T23:29:43.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb0900a5310 0x7fb090143200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0900b71e0 0x7fb09006aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb09013fce0 0x7fb09013cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09006b420 con 0x7fb09013fce0 2023-09-29T23:29:43.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09006b6e0 con 0x7fb0900b71e0 2023-09-29T23:29:43.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900a2bf0 con 0x7fb0900a5310 2023-09-29T23:29:43.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb08ed9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb0900a5310 0x7fb090143200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb08ed9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb0900a5310 0x7fb090143200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60336/0 (socket says 172.21.15.143:60336) 2023-09-29T23:29:43.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb08ed9d700 1 -- 172.21.15.143:0/235147166 learned_addr learned my addr 172.21.15.143:0/235147166 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:43.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb08f59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb09013fce0 0x7fb09013cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb08e59c700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0900b71e0 0x7fb09006aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb08ed9d700 1 -- 172.21.15.143:0/235147166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0900b71e0 msgr2=0x7fb09006aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb08ed9d700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0900b71e0 0x7fb09006aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb08ed9d700 1 -- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb09013fce0 msgr2=0x7fb09013cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb08ed9d700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb09013fce0 0x7fb09013cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb08ed9d700 1 -- 172.21.15.143:0/235147166 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078009420 con 0x7fb0900a5310 2023-09-29T23:29:43.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb08ed9d700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb0900a5310 0x7fb090143200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb078009fd0 tx=0x7fb078016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:43.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb086ffd700 1 -- 172.21.15.143:0/235147166 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07801cb80 con 0x7fb0900a5310 2023-09-29T23:29:43.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb086ffd700 1 -- 172.21.15.143:0/235147166 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07801cd00 con 0x7fb0900a5310 2023-09-29T23:29:43.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb086ffd700 1 -- 172.21.15.143:0/235147166 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07801a6a0 con 0x7fb0900a5310 2023-09-29T23:29:43.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 -- 172.21.15.143:0/235147166 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb090143740 con 0x7fb0900a5310 2023-09-29T23:29:43.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.712+0000 7fb0949ed700 1 -- 172.21.15.143:0/235147166 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb090143fe0 con 0x7fb0900a5310 2023-09-29T23:29:43.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.716+0000 7fb0949ed700 1 -- 172.21.15.143:0/235147166 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09005ecb0 con 0x7fb0900a5310 2023-09-29T23:29:43.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.716+0000 7fb086ffd700 1 -- 172.21.15.143:0/235147166 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fb07801aad0 con 0x7fb0900a5310 2023-09-29T23:29:43.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.716+0000 7fb086ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:43.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.716+0000 7fb086ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:43.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.716+0000 7fb086ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:43.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.716+0000 7fb086ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:43.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.716+0000 7fb086ffd700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb07408e8b0 0x7fb074090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:43.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.716+0000 7fb08e59c700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb07408e8b0 0x7fb074090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:43.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.716+0000 7fb086ffd700 1 -- 172.21.15.143:0/235147166 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7fb0780b6540 con 0x7fb0900a5310 2023-09-29T23:29:43.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.716+0000 7fb08e59c700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb07408e8b0 0x7fb074090d60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb080007fd0 tx=0x7fb080009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:43.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.720+0000 7fb086ffd700 1 -- 172.21.15.143:0/235147166 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb078087000 con 0x7fb0900a5310 2023-09-29T23:29:43.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.876+0000 7fb0949ed700 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2023-09-29T23:29:43.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.876+0000 7fb0949ed700 1 -- 172.21.15.143:0/235147166 --> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb09004b5c0 con 0x7fb07408e8b0 2023-09-29T23:29:43.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.884+0000 7fb086ffd700 1 -- 172.21.15.143:0/235147166 <== mgr.4101 v2:172.21.15.143:6808/27602 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+256577 (secure 0 0 0) 0x7fb09004b5c0 con 0x7fb07408e8b0 2023-09-29T23:29:43.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.884+0000 7fb086ffd700 20 mgrc handle_command_reply tid 0 r 0 2023-09-29T23:29:43.889 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 10 mgrc shutdown 2023-09-29T23:29:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 -- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb07408e8b0 msgr2=0x7fb074090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb07408e8b0 0x7fb074090d60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb080007fd0 tx=0x7fb080009040 comp rx=0 tx=0).stop 2023-09-29T23:29:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 -- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb0900a5310 msgr2=0x7fb090143200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb0900a5310 0x7fb090143200 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb078009fd0 tx=0x7fb078016000 comp rx=0 tx=0).stop 2023-09-29T23:29:43.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 -- 172.21.15.143:0/235147166 shutdown_connections 2023-09-29T23:29:43.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb07408e8b0 0x7fb074090d60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb09013fce0 0x7fb09013cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0900b71e0 0x7fb09006aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 --2- 172.21.15.143:0/235147166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb0900a5310 0x7fb090143200 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:43.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 -- 172.21.15.143:0/235147166 >> 172.21.15.143:0/235147166 conn(0x7fb090066c90 msgr2=0x7fb0900b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:43.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 -- 172.21.15.143:0/235147166 shutdown_connections 2023-09-29T23:29:43.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:43.888+0000 7fb0949ed700 1 -- 172.21.15.143:0/235147166 wait complete. 2023-09-29T23:29:43.892 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T23:29:43.902 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-09-29T23:29:42.462750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":46032,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":45376,"kb_avail":377441328,"statfs":{"total":386547056640,"available":386499919872,"internally_reserved":0,"allocated":638976,"data_stored":1335992,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":46465024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.233117"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":162,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.772100+0000","last_change":"2023-09-29T23:29:29.623917+0000","last_active":"2023-09-29T23:29:35.772100+0000","last_peered":"2023-09-29T23:29:35.772100+0000","last_clean":"2023-09-29T23:29:35.772100+0000","last_became_active":"2023-09-29T23:29:29.621470+0000","last_became_peered":"2023-09-29T23:29:29.621470+0000","last_unstale":"2023-09-29T23:29:35.772100+0000","last_undegraded":"2023-09-29T23:29:35.772100+0000","last_fullsized":"2023-09-29T23:29:35.772100+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_clean_scrub_stamp":"2023-09-29T23:29:28.522101+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:37:09.181891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100857+0000","last_change":"2023-09-29T23:29:35.099334+0000","last_active":"2023-09-29T23:29:35.100857+0000","last_peered":"2023-09-29T23:29:35.100857+0000","last_clean":"2023-09-29T23:29:35.100857+0000","last_became_active":"2023-09-29T23:29:35.098819+0000","last_became_peered":"2023-09-29T23:29:35.098819+0000","last_unstale":"2023-09-29T23:29:35.100857+0000","last_undegraded":"2023-09-29T23:29:35.100857+0000","last_fullsized":"2023-09-29T23:29:35.100857+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:29:21.699082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100800+0000","last_change":"2023-09-29T23:29:35.098951+0000","last_active":"2023-09-29T23:29:35.100800+0000","last_peered":"2023-09-29T23:29:35.100800+0000","last_clean":"2023-09-29T23:29:35.100800+0000","last_became_active":"2023-09-29T23:29:35.098555+0000","last_became_peered":"2023-09-29T23:29:35.098555+0000","last_unstale":"2023-09-29T23:29:35.100800+0000","last_undegraded":"2023-09-29T23:29:35.100800+0000","last_fullsized":"2023-09-29T23:29:35.100800+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:43:14.246995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":8,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.095306+0000","last_change":"2023-09-29T23:29:35.095306+0000","last_active":"2023-09-29T23:29:35.095306+0000","last_peered":"2023-09-29T23:29:35.095306+0000","last_clean":"2023-09-29T23:29:35.095306+0000","last_became_active":"2023-09-29T23:29:35.093309+0000","last_became_peered":"2023-09-29T23:29:35.093309+0000","last_unstale":"2023-09-29T23:29:35.095306+0000","last_undegraded":"2023-09-29T23:29:35.095306+0000","last_fullsized":"2023-09-29T23:29:35.095306+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:10:21.305466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100907+0000","last_change":"2023-09-29T23:29:35.099643+0000","last_active":"2023-09-29T23:29:35.100907+0000","last_peered":"2023-09-29T23:29:35.100907+0000","last_clean":"2023-09-29T23:29:35.100907+0000","last_became_active":"2023-09-29T23:29:35.099061+0000","last_became_peered":"2023-09-29T23:29:35.099061+0000","last_unstale":"2023-09-29T23:29:35.100907+0000","last_undegraded":"2023-09-29T23:29:35.100907+0000","last_fullsized":"2023-09-29T23:29:35.100907+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:30.050339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.102979+0000","last_change":"2023-09-29T23:29:35.100113+0000","last_active":"2023-09-29T23:29:35.102979+0000","last_peered":"2023-09-29T23:29:35.102979+0000","last_clean":"2023-09-29T23:29:35.102979+0000","last_became_active":"2023-09-29T23:29:35.099578+0000","last_became_peered":"2023-09-29T23:29:35.099578+0000","last_unstale":"2023-09-29T23:29:35.102979+0000","last_undegraded":"2023-09-29T23:29:35.102979+0000","last_fullsized":"2023-09-29T23:29:35.102979+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:57:39.508565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.100822+0000","last_change":"2023-09-29T23:29:35.099325+0000","last_active":"2023-09-29T23:29:35.100822+0000","last_peered":"2023-09-29T23:29:35.100822+0000","last_clean":"2023-09-29T23:29:35.100822+0000","last_became_active":"2023-09-29T23:29:35.098701+0000","last_became_peered":"2023-09-29T23:29:35.098701+0000","last_unstale":"2023-09-29T23:29:35.100822+0000","last_undegraded":"2023-09-29T23:29:35.100822+0000","last_fullsized":"2023-09-29T23:29:35.100822+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:35:07.769754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.098840+0000","last_change":"2023-09-29T23:29:35.096994+0000","last_active":"2023-09-29T23:29:35.098840+0000","last_peered":"2023-09-29T23:29:35.098840+0000","last_clean":"2023-09-29T23:29:35.098840+0000","last_became_active":"2023-09-29T23:29:35.096568+0000","last_became_peered":"2023-09-29T23:29:35.096568+0000","last_unstale":"2023-09-29T23:29:35.098840+0000","last_undegraded":"2023-09-29T23:29:35.098840+0000","last_fullsized":"2023-09-29T23:29:35.098840+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:22:06.698301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":14,"state":"active+clean","last_fresh":"2023-09-29T23:29:35.777899+0000","last_change":"2023-09-29T23:29:35.776424+0000","last_active":"2023-09-29T23:29:35.777899+0000","last_peered":"2023-09-29T23:29:35.777899+0000","last_clean":"2023-09-29T23:29:35.777899+0000","last_became_active":"2023-09-29T23:29:35.775625+0000","last_became_peered":"2023-09-29T23:29:35.775625+0000","last_unstale":"2023-09-29T23:29:35.777899+0000","last_undegraded":"2023-09-29T23:29:35.777899+0000","last_fullsized":"2023-09-29T23:29:35.777899+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:46.078517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:39.227089+0000","last_change":"2023-09-29T23:29:39.227089+0000","last_active":"2023-09-29T23:29:39.227089+0000","last_peered":"2023-09-29T23:29:39.227089+0000","last_clean":"2023-09-29T23:29:39.227089+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:39.227089+0000","last_undegraded":"2023-09-29T23:29:39.227089+0000","last_fullsized":"2023-09-29T23:29:39.227089+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-29T23:29:38.223331+0000","last_change":"2023-09-29T23:29:38.223331+0000","last_active":"2023-09-29T23:29:38.223331+0000","last_peered":"2023-09-29T23:29:38.223331+0000","last_clean":"2023-09-29T23:29:38.223331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-29T23:29:38.223331+0000","last_undegraded":"2023-09-29T23:29:38.223331+0000","last_fullsized":"2023-09-29T23:29:38.223331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12224,"kb_used_data":120,"kb_used_omap":0,"kb_used_meta":12096,"kb_avail":94359616,"statfs":{"total":96636764160,"available":96624246784,"internally_reserved":0,"allocated":122880,"data_stored":36810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12386304},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12800,"kb_used_data":184,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94359040,"statfs":{"total":96636764160,"available":96623656960,"internally_reserved":0,"allocated":188416,"data_stored":627178,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12808,"kb_used_data":192,"kb_used_omap":0,"kb_used_meta":12608,"kb_avail":94359032,"statfs":{"total":96636764160,"available":96623648768,"internally_reserved":0,"allocated":196608,"data_stored":631186,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":12910592},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8200,"kb_used_data":128,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363640,"statfs":{"total":96636764160,"available":96628367360,"internally_reserved":0,"allocated":131072,"data_stored":40818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T23:29:46.908 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T23:29:47.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f5063b73700 1 Processor -- start 2023-09-29T23:29:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f5063b73700 1 -- start start 2023-09-29T23:29:47.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f5063b73700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f505c0b3a40 0x7f505c0a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f5063b73700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f505c142180 0x7f505c134440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f5063b73700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f505c1418f0 con 0x7f505c141d80 2023-09-29T23:29:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f5063b73700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c0b3e10 con 0x7f505c0b3a40 2023-09-29T23:29:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f5063b73700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f505c0a8280 con 0x7f505c142180 2023-09-29T23:29:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f506190f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f505c0b3a40 0x7f505c0a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f506190f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f505c0b3a40 0x7f505c0a7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34460/0 (socket says 172.21.15.143:34460) 2023-09-29T23:29:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f506190f700 1 -- 172.21.15.143:0/919168908 learned_addr learned my addr 172.21.15.143:0/919168908 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:47.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f5062110700 1 --2- 172.21.15.143:0/919168908 >> v2:172.21.15.156:3300/0 conn(0x7f505c142180 0x7f505c134440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f506190f700 1 -- 172.21.15.143:0/919168908 >> v2:172.21.15.156:3300/0 conn(0x7f505c142180 msgr2=0x7f505c134440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f506190f700 1 --2- 172.21.15.143:0/919168908 >> v2:172.21.15.156:3300/0 conn(0x7f505c142180 0x7f505c134440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f506190f700 1 -- 172.21.15.143:0/919168908 >> v1:172.21.15.156:6789/0 conn(0x7f505c141d80 legacy=0x7f505c133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f506190f700 1 -- 172.21.15.143:0/919168908 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f505c0a4bc0 con 0x7f505c0b3a40 2023-09-29T23:29:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.020+0000 7f506190f700 1 --2- 172.21.15.143:0/919168908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505c0b3a40 0x7f505c0a7d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f505c131660 tx=0x7f5050009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977fb08da24419fe server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f506090d700 1 -- 172.21.15.143:0/919168908 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050009e70 con 0x7f505c0b3a40 2023-09-29T23:29:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f506090d700 1 -- 172.21.15.143:0/919168908 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505000b8f0 con 0x7f505c0b3a40 2023-09-29T23:29:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f506090d700 1 -- 172.21.15.143:0/919168908 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000bc90 con 0x7f505c0b3a40 2023-09-29T23:29:47.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 -- 172.21.15.143:0/919168908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505c0b3a40 msgr2=0x7f505c0a7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 --2- 172.21.15.143:0/919168908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505c0b3a40 0x7f505c0a7d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f505c131660 tx=0x7f5050009250 comp rx=0 tx=0).stop 2023-09-29T23:29:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 -- 172.21.15.143:0/919168908 shutdown_connections 2023-09-29T23:29:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 --2- 172.21.15.143:0/919168908 >> v2:172.21.15.156:3300/0 conn(0x7f505c142180 0x7f505c134440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 --2- 172.21.15.143:0/919168908 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505c0b3a40 0x7f505c0a7d40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 -- 172.21.15.143:0/919168908 >> 172.21.15.143:0/919168908 conn(0x7f505c069820 msgr2=0x7f505c069c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 -- 172.21.15.143:0/919168908 shutdown_connections 2023-09-29T23:29:47.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 -- 172.21.15.143:0/919168908 wait complete. 2023-09-29T23:29:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 Processor -- start 2023-09-29T23:29:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 -- start start 2023-09-29T23:29:47.028 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505c141d80 0x7f505c06cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f505c142180 0x7f505c0af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f505c0af610 0x7f505c150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c0afa80 con 0x7f505c141d80 2023-09-29T23:29:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505c151430 con 0x7f505c142180 2023-09-29T23:29:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5063b73700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f505c151170 con 0x7f505c0af610 2023-09-29T23:29:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f506190f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505c141d80 0x7f505c06cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f506190f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505c141d80 0x7f505c06cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34462/0 (socket says 172.21.15.143:34462) 2023-09-29T23:29:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f506190f700 1 -- 172.21.15.143:0/3948818202 learned_addr learned my addr 172.21.15.143:0/3948818202 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f5062110700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f505c0af610 0x7f505c150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.024+0000 7f506110e700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f505c142180 0x7f505c0af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f506190f700 1 -- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f505c0af610 msgr2=0x7f505c150420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f506190f700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f505c0af610 0x7f505c150420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f506190f700 1 -- 172.21.15.143:0/3948818202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f505c142180 msgr2=0x7f505c0af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f506190f700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f505c142180 0x7f505c0af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f506190f700 1 -- 172.21.15.143:0/3948818202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50500094f0 con 0x7f505c141d80 2023-09-29T23:29:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f506190f700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505c141d80 0x7f505c06cf50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5050006fd0 tx=0x7f5050017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f504dffb700 1 -- 172.21.15.143:0/3948818202 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505000bc70 con 0x7f505c141d80 2023-09-29T23:29:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f504dffb700 1 -- 172.21.15.143:0/3948818202 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505000be40 con 0x7f505c141d80 2023-09-29T23:29:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f504dffb700 1 -- 172.21.15.143:0/3948818202 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5050024670 con 0x7f505c141d80 2023-09-29T23:29:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f5063b73700 1 -- 172.21.15.143:0/3948818202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505c06d6c0 con 0x7f505c141d80 2023-09-29T23:29:47.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f5063b73700 1 -- 172.21.15.143:0/3948818202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505c06d880 con 0x7f505c141d80 2023-09-29T23:29:47.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.028+0000 7f5063b73700 1 -- 172.21.15.143:0/3948818202 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f505c05ecb0 con 0x7f505c141d80 2023-09-29T23:29:47.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.032+0000 7f504dffb700 1 -- 172.21.15.143:0/3948818202 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f505001a420 con 0x7f505c141d80 2023-09-29T23:29:47.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.032+0000 7f504dffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.032+0000 7f504dffb700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.032+0000 7f504dffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.032+0000 7f504dffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.032+0000 7f504dffb700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f504008e8b0 0x7f5040090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.032+0000 7f506110e700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f504008e8b0 0x7f5040090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.032+0000 7f504dffb700 1 -- 172.21.15.143:0/3948818202 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f50500b5af0 con 0x7f505c141d80 2023-09-29T23:29:47.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.032+0000 7f506110e700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f504008e8b0 0x7f5040090d60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5058007fd0 tx=0x7f5058009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:47.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.036+0000 7f504dffb700 1 -- 172.21.15.143:0/3948818202 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f50500865b0 con 0x7f505c141d80 2023-09-29T23:29:47.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.196+0000 7f5063b73700 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2023-09-29T23:29:47.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.196+0000 7f5063b73700 1 -- 172.21.15.143:0/3948818202 --> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f505c13fbc0 con 0x7f504008e8b0 2023-09-29T23:29:47.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.212+0000 7f504dffb700 1 -- 172.21.15.143:0/3948818202 <== mgr.4101 v2:172.21.15.143:6808/27602 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+269506 (secure 0 0 0) 0x7f505c13fbc0 con 0x7f504008e8b0 2023-09-29T23:29:47.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.212+0000 7f504dffb700 20 mgrc handle_command_reply tid 0 r 0 2023-09-29T23:29:47.219 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:47.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.216+0000 7f5063b73700 10 mgrc shutdown 2023-09-29T23:29:47.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.216+0000 7f5063b73700 1 -- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f504008e8b0 msgr2=0x7f5040090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.216+0000 7f5063b73700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f504008e8b0 0x7f5040090d60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5058007fd0 tx=0x7f5058009040 comp rx=0 tx=0).stop 2023-09-29T23:29:47.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.216+0000 7f5063b73700 1 -- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505c141d80 msgr2=0x7f505c06cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.216+0000 7f5063b73700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505c141d80 0x7f505c06cf50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5050006fd0 tx=0x7f5050017000 comp rx=0 tx=0).stop 2023-09-29T23:29:47.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.220+0000 7f5063b73700 1 -- 172.21.15.143:0/3948818202 shutdown_connections 2023-09-29T23:29:47.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.220+0000 7f5063b73700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f504008e8b0 0x7f5040090d60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.220+0000 7f5063b73700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f505c0af610 0x7f505c150420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.220+0000 7f5063b73700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f505c142180 0x7f505c0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.220+0000 7f5063b73700 1 --2- 172.21.15.143:0/3948818202 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f505c141d80 0x7f505c06cf50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.220+0000 7f5063b73700 1 -- 172.21.15.143:0/3948818202 >> 172.21.15.143:0/3948818202 conn(0x7f505c069820 msgr2=0x7f505c067480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:47.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.220+0000 7f5063b73700 1 -- 172.21.15.143:0/3948818202 shutdown_connections 2023-09-29T23:29:47.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.220+0000 7f5063b73700 1 -- 172.21.15.143:0/3948818202 wait complete. 2023-09-29T23:29:47.223 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T23:29:47.236 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-09-29T23:29:46.464199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":47376,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":377439984,"statfs":{"total":386547056640,"available":386498543616,"internally_reserved":0,"allocated":901120,"data_stored":1426840,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.230028"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.639872+0000","last_change":"2023-09-29T23:29:39.634761+0000","last_active":"2023-09-29T23:29:39.639872+0000","last_peered":"2023-09-29T23:29:39.639872+0000","last_clean":"2023-09-29T23:29:39.639872+0000","last_became_active":"2023-09-29T23:29:39.634501+0000","last_became_peered":"2023-09-29T23:29:39.634501+0000","last_unstale":"2023-09-29T23:29:39.639872+0000","last_undegraded":"2023-09-29T23:29:39.639872+0000","last_fullsized":"2023-09-29T23:29:39.639872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:20:37.281771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620836+0000","last_change":"2023-09-29T23:29:39.615851+0000","last_active":"2023-09-29T23:29:39.620836+0000","last_peered":"2023-09-29T23:29:39.620836+0000","last_clean":"2023-09-29T23:29:39.620836+0000","last_became_active":"2023-09-29T23:29:39.615521+0000","last_became_peered":"2023-09-29T23:29:39.615521+0000","last_unstale":"2023-09-29T23:29:39.620836+0000","last_undegraded":"2023-09-29T23:29:39.620836+0000","last_fullsized":"2023-09-29T23:29:39.620836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:17:42.897947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647121+0000","last_change":"2023-09-29T23:29:39.636680+0000","last_active":"2023-09-29T23:29:39.647121+0000","last_peered":"2023-09-29T23:29:39.647121+0000","last_clean":"2023-09-29T23:29:39.647121+0000","last_became_active":"2023-09-29T23:29:39.634721+0000","last_became_peered":"2023-09-29T23:29:39.634721+0000","last_unstale":"2023-09-29T23:29:39.647121+0000","last_undegraded":"2023-09-29T23:29:39.647121+0000","last_fullsized":"2023-09-29T23:29:39.647121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:36.503704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625198+0000","last_change":"2023-09-29T23:29:39.612565+0000","last_active":"2023-09-29T23:29:39.625198+0000","last_peered":"2023-09-29T23:29:39.625198+0000","last_clean":"2023-09-29T23:29:39.625198+0000","last_became_active":"2023-09-29T23:29:39.612183+0000","last_became_peered":"2023-09-29T23:29:39.612183+0000","last_unstale":"2023-09-29T23:29:39.625198+0000","last_undegraded":"2023-09-29T23:29:39.625198+0000","last_fullsized":"2023-09-29T23:29:39.625198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:42.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645186+0000","last_change":"2023-09-29T23:29:39.633553+0000","last_active":"2023-09-29T23:29:39.645186+0000","last_peered":"2023-09-29T23:29:39.645186+0000","last_clean":"2023-09-29T23:29:39.645186+0000","last_became_active":"2023-09-29T23:29:39.633127+0000","last_became_peered":"2023-09-29T23:29:39.633127+0000","last_unstale":"2023-09-29T23:29:39.645186+0000","last_undegraded":"2023-09-29T23:29:39.645186+0000","last_fullsized":"2023-09-29T23:29:39.645186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:47:58.354132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645627+0000","last_change":"2023-09-29T23:29:39.636332+0000","last_active":"2023-09-29T23:29:39.645627+0000","last_peered":"2023-09-29T23:29:39.645627+0000","last_clean":"2023-09-29T23:29:39.645627+0000","last_became_active":"2023-09-29T23:29:39.636025+0000","last_became_peered":"2023-09-29T23:29:39.636025+0000","last_unstale":"2023-09-29T23:29:39.645627+0000","last_undegraded":"2023-09-29T23:29:39.645627+0000","last_fullsized":"2023-09-29T23:29:39.645627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:33.457443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645224+0000","last_change":"2023-09-29T23:29:39.634398+0000","last_active":"2023-09-29T23:29:39.645224+0000","last_peered":"2023-09-29T23:29:39.645224+0000","last_clean":"2023-09-29T23:29:39.645224+0000","last_became_active":"2023-09-29T23:29:39.633340+0000","last_became_peered":"2023-09-29T23:29:39.633340+0000","last_unstale":"2023-09-29T23:29:39.645224+0000","last_undegraded":"2023-09-29T23:29:39.645224+0000","last_fullsized":"2023-09-29T23:29:39.645224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:33:09.568111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629860+0000","last_change":"2023-09-29T23:29:39.618059+0000","last_active":"2023-09-29T23:29:39.629860+0000","last_peered":"2023-09-29T23:29:39.629860+0000","last_clean":"2023-09-29T23:29:39.629860+0000","last_became_active":"2023-09-29T23:29:39.617767+0000","last_became_peered":"2023-09-29T23:29:39.617767+0000","last_unstale":"2023-09-29T23:29:39.629860+0000","last_undegraded":"2023-09-29T23:29:39.629860+0000","last_fullsized":"2023-09-29T23:29:39.629860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:55:52.762563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.630737+0000","last_change":"2023-09-29T23:29:39.626861+0000","last_active":"2023-09-29T23:29:39.630737+0000","last_peered":"2023-09-29T23:29:39.630737+0000","last_clean":"2023-09-29T23:29:39.630737+0000","last_became_active":"2023-09-29T23:29:39.626482+0000","last_became_peered":"2023-09-29T23:29:39.626482+0000","last_unstale":"2023-09-29T23:29:39.630737+0000","last_undegraded":"2023-09-29T23:29:39.630737+0000","last_fullsized":"2023-09-29T23:29:39.630737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:48.157072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641519+0000","last_change":"2023-09-29T23:29:39.640158+0000","last_active":"2023-09-29T23:29:39.641519+0000","last_peered":"2023-09-29T23:29:39.641519+0000","last_clean":"2023-09-29T23:29:39.641519+0000","last_became_active":"2023-09-29T23:29:39.639864+0000","last_became_peered":"2023-09-29T23:29:39.639864+0000","last_unstale":"2023-09-29T23:29:39.641519+0000","last_undegraded":"2023-09-29T23:29:39.641519+0000","last_fullsized":"2023-09-29T23:29:39.641519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:46:27.931849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625089+0000","last_change":"2023-09-29T23:29:39.612352+0000","last_active":"2023-09-29T23:29:39.625089+0000","last_peered":"2023-09-29T23:29:39.625089+0000","last_clean":"2023-09-29T23:29:39.625089+0000","last_became_active":"2023-09-29T23:29:39.611961+0000","last_became_peered":"2023-09-29T23:29:39.611961+0000","last_unstale":"2023-09-29T23:29:39.625089+0000","last_undegraded":"2023-09-29T23:29:39.625089+0000","last_fullsized":"2023-09-29T23:29:39.625089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:46:32.838187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638115+0000","last_change":"2023-09-29T23:29:39.632780+0000","last_active":"2023-09-29T23:29:39.638115+0000","last_peered":"2023-09-29T23:29:39.638115+0000","last_clean":"2023-09-29T23:29:39.638115+0000","last_became_active":"2023-09-29T23:29:39.632492+0000","last_became_peered":"2023-09-29T23:29:39.632492+0000","last_unstale":"2023-09-29T23:29:39.638115+0000","last_undegraded":"2023-09-29T23:29:39.638115+0000","last_fullsized":"2023-09-29T23:29:39.638115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:14:21.471042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.636401+0000","last_change":"2023-09-29T23:29:39.633421+0000","last_active":"2023-09-29T23:29:39.636401+0000","last_peered":"2023-09-29T23:29:39.636401+0000","last_clean":"2023-09-29T23:29:39.636401+0000","last_became_active":"2023-09-29T23:29:39.633127+0000","last_became_peered":"2023-09-29T23:29:39.633127+0000","last_unstale":"2023-09-29T23:29:39.636401+0000","last_undegraded":"2023-09-29T23:29:39.636401+0000","last_fullsized":"2023-09-29T23:29:39.636401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:58:35.776573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629920+0000","last_change":"2023-09-29T23:29:39.621931+0000","last_active":"2023-09-29T23:29:39.629920+0000","last_peered":"2023-09-29T23:29:39.629920+0000","last_clean":"2023-09-29T23:29:39.629920+0000","last_became_active":"2023-09-29T23:29:39.621616+0000","last_became_peered":"2023-09-29T23:29:39.621616+0000","last_unstale":"2023-09-29T23:29:39.629920+0000","last_undegraded":"2023-09-29T23:29:39.629920+0000","last_fullsized":"2023-09-29T23:29:39.629920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:53:43.483894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629993+0000","last_change":"2023-09-29T23:29:39.622200+0000","last_active":"2023-09-29T23:29:39.629993+0000","last_peered":"2023-09-29T23:29:39.629993+0000","last_clean":"2023-09-29T23:29:39.629993+0000","last_became_active":"2023-09-29T23:29:39.621385+0000","last_became_peered":"2023-09-29T23:29:39.621385+0000","last_unstale":"2023-09-29T23:29:39.629993+0000","last_undegraded":"2023-09-29T23:29:39.629993+0000","last_fullsized":"2023-09-29T23:29:39.629993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:17:19.555819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645817+0000","last_change":"2023-09-29T23:29:39.638607+0000","last_active":"2023-09-29T23:29:39.645817+0000","last_peered":"2023-09-29T23:29:39.645817+0000","last_clean":"2023-09-29T23:29:39.645817+0000","last_became_active":"2023-09-29T23:29:39.635921+0000","last_became_peered":"2023-09-29T23:29:39.635921+0000","last_unstale":"2023-09-29T23:29:39.645817+0000","last_undegraded":"2023-09-29T23:29:39.645817+0000","last_fullsized":"2023-09-29T23:29:39.645817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:27:52.497036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.631406+0000","last_change":"2023-09-29T23:29:39.622234+0000","last_active":"2023-09-29T23:29:39.631406+0000","last_peered":"2023-09-29T23:29:39.631406+0000","last_clean":"2023-09-29T23:29:39.631406+0000","last_became_active":"2023-09-29T23:29:39.621861+0000","last_became_peered":"2023-09-29T23:29:39.621861+0000","last_unstale":"2023-09-29T23:29:39.631406+0000","last_undegraded":"2023-09-29T23:29:39.631406+0000","last_fullsized":"2023-09-29T23:29:39.631406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:54:29.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.646545+0000","last_change":"2023-09-29T23:29:39.639501+0000","last_active":"2023-09-29T23:29:39.646545+0000","last_peered":"2023-09-29T23:29:39.646545+0000","last_clean":"2023-09-29T23:29:39.646545+0000","last_became_active":"2023-09-29T23:29:39.638774+0000","last_became_peered":"2023-09-29T23:29:39.638774+0000","last_unstale":"2023-09-29T23:29:39.646545+0000","last_undegraded":"2023-09-29T23:29:39.646545+0000","last_fullsized":"2023-09-29T23:29:39.646545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:37:30.614407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.636591+0000","last_change":"2023-09-29T23:29:39.633491+0000","last_active":"2023-09-29T23:29:39.636591+0000","last_peered":"2023-09-29T23:29:39.636591+0000","last_clean":"2023-09-29T23:29:39.636591+0000","last_became_active":"2023-09-29T23:29:39.633174+0000","last_became_peered":"2023-09-29T23:29:39.633174+0000","last_unstale":"2023-09-29T23:29:39.636591+0000","last_undegraded":"2023-09-29T23:29:39.636591+0000","last_fullsized":"2023-09-29T23:29:39.636591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:36:55.020338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625219+0000","last_change":"2023-09-29T23:29:39.613672+0000","last_active":"2023-09-29T23:29:39.625219+0000","last_peered":"2023-09-29T23:29:39.625219+0000","last_clean":"2023-09-29T23:29:39.625219+0000","last_became_active":"2023-09-29T23:29:39.612667+0000","last_became_peered":"2023-09-29T23:29:39.612667+0000","last_unstale":"2023-09-29T23:29:39.625219+0000","last_undegraded":"2023-09-29T23:29:39.625219+0000","last_fullsized":"2023-09-29T23:29:39.625219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:47:46.021660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641070+0000","last_change":"2023-09-29T23:29:39.639206+0000","last_active":"2023-09-29T23:29:39.641070+0000","last_peered":"2023-09-29T23:29:39.641070+0000","last_clean":"2023-09-29T23:29:39.641070+0000","last_became_active":"2023-09-29T23:29:39.638930+0000","last_became_peered":"2023-09-29T23:29:39.638930+0000","last_unstale":"2023-09-29T23:29:39.641070+0000","last_undegraded":"2023-09-29T23:29:39.641070+0000","last_fullsized":"2023-09-29T23:29:39.641070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:34:56.298163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629896+0000","last_change":"2023-09-29T23:29:39.621733+0000","last_active":"2023-09-29T23:29:39.629896+0000","last_peered":"2023-09-29T23:29:39.629896+0000","last_clean":"2023-09-29T23:29:39.629896+0000","last_became_active":"2023-09-29T23:29:39.621410+0000","last_became_peered":"2023-09-29T23:29:39.621410+0000","last_unstale":"2023-09-29T23:29:39.629896+0000","last_undegraded":"2023-09-29T23:29:39.629896+0000","last_fullsized":"2023-09-29T23:29:39.629896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:13:11.017795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645339+0000","last_change":"2023-09-29T23:29:39.635888+0000","last_active":"2023-09-29T23:29:39.645339+0000","last_peered":"2023-09-29T23:29:39.645339+0000","last_clean":"2023-09-29T23:29:39.645339+0000","last_became_active":"2023-09-29T23:29:39.634120+0000","last_became_peered":"2023-09-29T23:29:39.634120+0000","last_unstale":"2023-09-29T23:29:39.645339+0000","last_undegraded":"2023-09-29T23:29:39.645339+0000","last_fullsized":"2023-09-29T23:29:39.645339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:16:01.339180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625208+0000","last_change":"2023-09-29T23:29:39.612568+0000","last_active":"2023-09-29T23:29:39.625208+0000","last_peered":"2023-09-29T23:29:39.625208+0000","last_clean":"2023-09-29T23:29:39.625208+0000","last_became_active":"2023-09-29T23:29:39.612349+0000","last_became_peered":"2023-09-29T23:29:39.612349+0000","last_unstale":"2023-09-29T23:29:39.625208+0000","last_undegraded":"2023-09-29T23:29:39.625208+0000","last_fullsized":"2023-09-29T23:29:39.625208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:36:55.020338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641527+0000","last_change":"2023-09-29T23:29:39.640244+0000","last_active":"2023-09-29T23:29:39.641527+0000","last_peered":"2023-09-29T23:29:39.641527+0000","last_clean":"2023-09-29T23:29:39.641527+0000","last_became_active":"2023-09-29T23:29:39.639928+0000","last_became_peered":"2023-09-29T23:29:39.639928+0000","last_unstale":"2023-09-29T23:29:39.641527+0000","last_undegraded":"2023-09-29T23:29:39.641527+0000","last_fullsized":"2023-09-29T23:29:39.641527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:20:35.058663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.610083+0000","last_change":"2023-09-29T23:29:39.609392+0000","last_active":"2023-09-29T23:29:39.610083+0000","last_peered":"2023-09-29T23:29:39.610083+0000","last_clean":"2023-09-29T23:29:39.610083+0000","last_became_active":"2023-09-29T23:29:39.609137+0000","last_became_peered":"2023-09-29T23:29:39.609137+0000","last_unstale":"2023-09-29T23:29:39.610083+0000","last_undegraded":"2023-09-29T23:29:39.610083+0000","last_fullsized":"2023-09-29T23:29:39.610083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:14:21.471042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647538+0000","last_change":"2023-09-29T23:29:39.646351+0000","last_active":"2023-09-29T23:29:39.647538+0000","last_peered":"2023-09-29T23:29:39.647538+0000","last_clean":"2023-09-29T23:29:39.647538+0000","last_became_active":"2023-09-29T23:29:39.646091+0000","last_became_peered":"2023-09-29T23:29:39.646091+0000","last_unstale":"2023-09-29T23:29:39.647538+0000","last_undegraded":"2023-09-29T23:29:39.647538+0000","last_fullsized":"2023-09-29T23:29:39.647538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:41:52.519640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.606960+0000","last_change":"2023-09-29T23:29:39.606266+0000","last_active":"2023-09-29T23:29:39.606960+0000","last_peered":"2023-09-29T23:29:39.606960+0000","last_clean":"2023-09-29T23:29:39.606960+0000","last_became_active":"2023-09-29T23:29:39.606025+0000","last_became_peered":"2023-09-29T23:29:39.606025+0000","last_unstale":"2023-09-29T23:29:39.606960+0000","last_undegraded":"2023-09-29T23:29:39.606960+0000","last_fullsized":"2023-09-29T23:29:39.606960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:17:42.897947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637537+0000","last_change":"2023-09-29T23:29:39.634587+0000","last_active":"2023-09-29T23:29:39.637537+0000","last_peered":"2023-09-29T23:29:39.637537+0000","last_clean":"2023-09-29T23:29:39.637537+0000","last_became_active":"2023-09-29T23:29:39.634344+0000","last_became_peered":"2023-09-29T23:29:39.634344+0000","last_unstale":"2023-09-29T23:29:39.637537+0000","last_undegraded":"2023-09-29T23:29:39.637537+0000","last_fullsized":"2023-09-29T23:29:39.637537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:20:37.281771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637839+0000","last_change":"2023-09-29T23:29:39.632991+0000","last_active":"2023-09-29T23:29:39.637839+0000","last_peered":"2023-09-29T23:29:39.637839+0000","last_clean":"2023-09-29T23:29:39.637839+0000","last_became_active":"2023-09-29T23:29:39.632721+0000","last_became_peered":"2023-09-29T23:29:39.632721+0000","last_unstale":"2023-09-29T23:29:39.637839+0000","last_undegraded":"2023-09-29T23:29:39.637839+0000","last_fullsized":"2023-09-29T23:29:39.637839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:25:55.111560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.618030+0000","last_change":"2023-09-29T23:29:39.616390+0000","last_active":"2023-09-29T23:29:39.618030+0000","last_peered":"2023-09-29T23:29:39.618030+0000","last_clean":"2023-09-29T23:29:39.618030+0000","last_became_active":"2023-09-29T23:29:39.615974+0000","last_became_peered":"2023-09-29T23:29:39.615974+0000","last_unstale":"2023-09-29T23:29:39.618030+0000","last_undegraded":"2023-09-29T23:29:39.618030+0000","last_fullsized":"2023-09-29T23:29:39.618030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:05:52.183625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.630420+0000","last_change":"2023-09-29T23:29:39.625067+0000","last_active":"2023-09-29T23:29:39.630420+0000","last_peered":"2023-09-29T23:29:39.630420+0000","last_clean":"2023-09-29T23:29:39.630420+0000","last_became_active":"2023-09-29T23:29:39.624793+0000","last_became_peered":"2023-09-29T23:29:39.624793+0000","last_unstale":"2023-09-29T23:29:39.630420+0000","last_undegraded":"2023-09-29T23:29:39.630420+0000","last_fullsized":"2023-09-29T23:29:39.630420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:48.157072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645155+0000","last_change":"2023-09-29T23:29:39.633681+0000","last_active":"2023-09-29T23:29:39.645155+0000","last_peered":"2023-09-29T23:29:39.645155+0000","last_clean":"2023-09-29T23:29:39.645155+0000","last_became_active":"2023-09-29T23:29:39.633400+0000","last_became_peered":"2023-09-29T23:29:39.633400+0000","last_unstale":"2023-09-29T23:29:39.645155+0000","last_undegraded":"2023-09-29T23:29:39.645155+0000","last_fullsized":"2023-09-29T23:29:39.645155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:29:56.190518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638194+0000","last_change":"2023-09-29T23:29:39.272166+0000","last_active":"2023-09-29T23:29:39.638194+0000","last_peered":"2023-09-29T23:29:39.638194+0000","last_clean":"2023-09-29T23:29:39.638194+0000","last_became_active":"2023-09-29T23:29:39.271616+0000","last_became_peered":"2023-09-29T23:29:39.271616+0000","last_unstale":"2023-09-29T23:29:39.638194+0000","last_undegraded":"2023-09-29T23:29:39.638194+0000","last_fullsized":"2023-09-29T23:29:39.638194+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:14:20.467285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647140+0000","last_change":"2023-09-29T23:29:39.639338+0000","last_active":"2023-09-29T23:29:39.647140+0000","last_peered":"2023-09-29T23:29:39.647140+0000","last_clean":"2023-09-29T23:29:39.647140+0000","last_became_active":"2023-09-29T23:29:39.639030+0000","last_became_peered":"2023-09-29T23:29:39.639030+0000","last_unstale":"2023-09-29T23:29:39.647140+0000","last_undegraded":"2023-09-29T23:29:39.647140+0000","last_fullsized":"2023-09-29T23:29:39.647140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:46:49.801617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.598222+0000","last_change":"2023-09-29T23:29:39.269808+0000","last_active":"2023-09-29T23:29:39.598222+0000","last_peered":"2023-09-29T23:29:39.598222+0000","last_clean":"2023-09-29T23:29:39.598222+0000","last_became_active":"2023-09-29T23:29:39.269509+0000","last_became_peered":"2023-09-29T23:29:39.269509+0000","last_unstale":"2023-09-29T23:29:39.598222+0000","last_undegraded":"2023-09-29T23:29:39.598222+0000","last_fullsized":"2023-09-29T23:29:39.598222+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:47:31.314319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.646474+0000","last_change":"2023-09-29T23:29:39.639099+0000","last_active":"2023-09-29T23:29:39.646474+0000","last_peered":"2023-09-29T23:29:39.646474+0000","last_clean":"2023-09-29T23:29:39.646474+0000","last_became_active":"2023-09-29T23:29:39.636608+0000","last_became_peered":"2023-09-29T23:29:39.636608+0000","last_unstale":"2023-09-29T23:29:39.646474+0000","last_undegraded":"2023-09-29T23:29:39.646474+0000","last_fullsized":"2023-09-29T23:29:39.646474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:24:20.819335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620024+0000","last_change":"2023-09-29T23:29:39.273761+0000","last_active":"2023-09-29T23:29:39.620024+0000","last_peered":"2023-09-29T23:29:39.620024+0000","last_clean":"2023-09-29T23:29:39.620024+0000","last_became_active":"2023-09-29T23:29:39.273376+0000","last_became_peered":"2023-09-29T23:29:39.273376+0000","last_unstale":"2023-09-29T23:29:39.620024+0000","last_undegraded":"2023-09-29T23:29:39.620024+0000","last_fullsized":"2023-09-29T23:29:39.620024+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:59:21.890466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.636454+0000","last_change":"2023-09-29T23:29:39.631948+0000","last_active":"2023-09-29T23:29:39.636454+0000","last_peered":"2023-09-29T23:29:39.636454+0000","last_clean":"2023-09-29T23:29:39.636454+0000","last_became_active":"2023-09-29T23:29:39.631531+0000","last_became_peered":"2023-09-29T23:29:39.631531+0000","last_unstale":"2023-09-29T23:29:39.636454+0000","last_undegraded":"2023-09-29T23:29:39.636454+0000","last_fullsized":"2023-09-29T23:29:39.636454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:04:09.469416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645828+0000","last_change":"2023-09-29T23:29:39.259965+0000","last_active":"2023-09-29T23:29:39.645828+0000","last_peered":"2023-09-29T23:29:39.645828+0000","last_clean":"2023-09-29T23:29:39.645828+0000","last_became_active":"2023-09-29T23:29:39.259617+0000","last_became_peered":"2023-09-29T23:29:39.259617+0000","last_unstale":"2023-09-29T23:29:39.645828+0000","last_undegraded":"2023-09-29T23:29:39.645828+0000","last_fullsized":"2023-09-29T23:29:39.645828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:43:32.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.630444+0000","last_change":"2023-09-29T23:29:39.262801+0000","last_active":"2023-09-29T23:29:39.630444+0000","last_peered":"2023-09-29T23:29:39.630444+0000","last_clean":"2023-09-29T23:29:39.630444+0000","last_became_active":"2023-09-29T23:29:39.259692+0000","last_became_peered":"2023-09-29T23:29:39.259692+0000","last_unstale":"2023-09-29T23:29:39.630444+0000","last_undegraded":"2023-09-29T23:29:39.630444+0000","last_fullsized":"2023-09-29T23:29:39.630444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:43:32.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645777+0000","last_change":"2023-09-29T23:29:39.636515+0000","last_active":"2023-09-29T23:29:39.645777+0000","last_peered":"2023-09-29T23:29:39.645777+0000","last_clean":"2023-09-29T23:29:39.645777+0000","last_became_active":"2023-09-29T23:29:39.633532+0000","last_became_peered":"2023-09-29T23:29:39.633532+0000","last_unstale":"2023-09-29T23:29:39.645777+0000","last_undegraded":"2023-09-29T23:29:39.645777+0000","last_fullsized":"2023-09-29T23:29:39.645777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:53:43.483894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625041+0000","last_change":"2023-09-29T23:29:39.268225+0000","last_active":"2023-09-29T23:29:39.625041+0000","last_peered":"2023-09-29T23:29:39.625041+0000","last_clean":"2023-09-29T23:29:39.625041+0000","last_became_active":"2023-09-29T23:29:39.267910+0000","last_became_peered":"2023-09-29T23:29:39.267910+0000","last_unstale":"2023-09-29T23:29:39.625041+0000","last_undegraded":"2023-09-29T23:29:39.625041+0000","last_fullsized":"2023-09-29T23:29:39.625041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:58:28.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.624786+0000","last_change":"2023-09-29T23:29:39.612447+0000","last_active":"2023-09-29T23:29:39.624786+0000","last_peered":"2023-09-29T23:29:39.624786+0000","last_clean":"2023-09-29T23:29:39.624786+0000","last_became_active":"2023-09-29T23:29:39.611905+0000","last_became_peered":"2023-09-29T23:29:39.611905+0000","last_unstale":"2023-09-29T23:29:39.624786+0000","last_undegraded":"2023-09-29T23:29:39.624786+0000","last_fullsized":"2023-09-29T23:29:39.624786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:31.379677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645233+0000","last_change":"2023-09-29T23:29:39.270818+0000","last_active":"2023-09-29T23:29:39.645233+0000","last_peered":"2023-09-29T23:29:39.645233+0000","last_clean":"2023-09-29T23:29:39.645233+0000","last_became_active":"2023-09-29T23:29:39.270419+0000","last_became_peered":"2023-09-29T23:29:39.270419+0000","last_unstale":"2023-09-29T23:29:39.645233+0000","last_undegraded":"2023-09-29T23:29:39.645233+0000","last_fullsized":"2023-09-29T23:29:39.645233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:39:03.054065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638105+0000","last_change":"2023-09-29T23:29:39.634163+0000","last_active":"2023-09-29T23:29:39.638105+0000","last_peered":"2023-09-29T23:29:39.638105+0000","last_clean":"2023-09-29T23:29:39.638105+0000","last_became_active":"2023-09-29T23:29:39.633902+0000","last_became_peered":"2023-09-29T23:29:39.633902+0000","last_unstale":"2023-09-29T23:29:39.638105+0000","last_undegraded":"2023-09-29T23:29:39.638105+0000","last_fullsized":"2023-09-29T23:29:39.638105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:48.157072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.616087+0000","last_change":"2023-09-29T23:29:39.267422+0000","last_active":"2023-09-29T23:29:39.616087+0000","last_peered":"2023-09-29T23:29:39.616087+0000","last_clean":"2023-09-29T23:29:39.616087+0000","last_became_active":"2023-09-29T23:29:39.267119+0000","last_became_peered":"2023-09-29T23:29:39.267119+0000","last_unstale":"2023-09-29T23:29:39.616087+0000","last_undegraded":"2023-09-29T23:29:39.616087+0000","last_fullsized":"2023-09-29T23:29:39.616087+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:29:06.658217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609399+0000","last_change":"2023-09-29T23:29:39.607523+0000","last_active":"2023-09-29T23:29:39.609399+0000","last_peered":"2023-09-29T23:29:39.609399+0000","last_clean":"2023-09-29T23:29:39.609399+0000","last_became_active":"2023-09-29T23:29:39.607172+0000","last_became_peered":"2023-09-29T23:29:39.607172+0000","last_unstale":"2023-09-29T23:29:39.609399+0000","last_undegraded":"2023-09-29T23:29:39.609399+0000","last_fullsized":"2023-09-29T23:29:39.609399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:05:52.183625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.614202+0000","last_change":"2023-09-29T23:29:39.259718+0000","last_active":"2023-09-29T23:29:39.614202+0000","last_peered":"2023-09-29T23:29:39.614202+0000","last_clean":"2023-09-29T23:29:39.614202+0000","last_became_active":"2023-09-29T23:29:39.259429+0000","last_became_peered":"2023-09-29T23:29:39.259429+0000","last_unstale":"2023-09-29T23:29:39.614202+0000","last_undegraded":"2023-09-29T23:29:39.614202+0000","last_fullsized":"2023-09-29T23:29:39.614202+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:18:38.314574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.635728+0000","last_change":"2023-09-29T23:29:39.631644+0000","last_active":"2023-09-29T23:29:39.635728+0000","last_peered":"2023-09-29T23:29:39.635728+0000","last_clean":"2023-09-29T23:29:39.635728+0000","last_became_active":"2023-09-29T23:29:39.630542+0000","last_became_peered":"2023-09-29T23:29:39.630542+0000","last_unstale":"2023-09-29T23:29:39.635728+0000","last_undegraded":"2023-09-29T23:29:39.635728+0000","last_fullsized":"2023-09-29T23:29:39.635728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:40:30.851104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.613760+0000","last_change":"2023-09-29T23:29:39.268889+0000","last_active":"2023-09-29T23:29:39.613760+0000","last_peered":"2023-09-29T23:29:39.613760+0000","last_clean":"2023-09-29T23:29:39.613760+0000","last_became_active":"2023-09-29T23:29:39.268443+0000","last_became_peered":"2023-09-29T23:29:39.268443+0000","last_unstale":"2023-09-29T23:29:39.613760+0000","last_undegraded":"2023-09-29T23:29:39.613760+0000","last_fullsized":"2023-09-29T23:29:39.613760+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:58:34.772816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647157+0000","last_change":"2023-09-29T23:29:39.639561+0000","last_active":"2023-09-29T23:29:39.647157+0000","last_peered":"2023-09-29T23:29:39.647157+0000","last_clean":"2023-09-29T23:29:39.647157+0000","last_became_active":"2023-09-29T23:29:39.639082+0000","last_became_peered":"2023-09-29T23:29:39.639082+0000","last_unstale":"2023-09-29T23:29:39.647157+0000","last_undegraded":"2023-09-29T23:29:39.647157+0000","last_fullsized":"2023-09-29T23:29:39.647157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:15:31.512106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620049+0000","last_change":"2023-09-29T23:29:39.274677+0000","last_active":"2023-09-29T23:29:39.620049+0000","last_peered":"2023-09-29T23:29:39.620049+0000","last_clean":"2023-09-29T23:29:39.620049+0000","last_became_active":"2023-09-29T23:29:39.274395+0000","last_became_peered":"2023-09-29T23:29:39.274395+0000","last_unstale":"2023-09-29T23:29:39.620049+0000","last_undegraded":"2023-09-29T23:29:39.620049+0000","last_fullsized":"2023-09-29T23:29:39.620049+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:48.655823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638074+0000","last_change":"2023-09-29T23:29:39.633620+0000","last_active":"2023-09-29T23:29:39.638074+0000","last_peered":"2023-09-29T23:29:39.638074+0000","last_clean":"2023-09-29T23:29:39.638074+0000","last_became_active":"2023-09-29T23:29:39.633346+0000","last_became_peered":"2023-09-29T23:29:39.633346+0000","last_unstale":"2023-09-29T23:29:39.638074+0000","last_undegraded":"2023-09-29T23:29:39.638074+0000","last_fullsized":"2023-09-29T23:29:39.638074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:59:22.894223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617510+0000","last_change":"2023-09-29T23:29:39.269804+0000","last_active":"2023-09-29T23:29:39.617510+0000","last_peered":"2023-09-29T23:29:39.617510+0000","last_clean":"2023-09-29T23:29:39.617510+0000","last_became_active":"2023-09-29T23:29:39.269530+0000","last_became_peered":"2023-09-29T23:29:39.269530+0000","last_unstale":"2023-09-29T23:29:39.617510+0000","last_undegraded":"2023-09-29T23:29:39.617510+0000","last_fullsized":"2023-09-29T23:29:39.617510+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:47:31.314319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647052+0000","last_change":"2023-09-29T23:29:39.645616+0000","last_active":"2023-09-29T23:29:39.647052+0000","last_peered":"2023-09-29T23:29:39.647052+0000","last_clean":"2023-09-29T23:29:39.647052+0000","last_became_active":"2023-09-29T23:29:39.644961+0000","last_became_peered":"2023-09-29T23:29:39.644961+0000","last_unstale":"2023-09-29T23:29:39.647052+0000","last_undegraded":"2023-09-29T23:29:39.647052+0000","last_fullsized":"2023-09-29T23:29:39.647052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:56:40.493860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":166,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.612078+0000","last_change":"2023-09-29T23:29:29.623917+0000","last_active":"2023-09-29T23:29:39.612078+0000","last_peered":"2023-09-29T23:29:39.612078+0000","last_clean":"2023-09-29T23:29:39.612078+0000","last_became_active":"2023-09-29T23:29:29.621470+0000","last_became_peered":"2023-09-29T23:29:29.621470+0000","last_unstale":"2023-09-29T23:29:39.612078+0000","last_undegraded":"2023-09-29T23:29:39.612078+0000","last_fullsized":"2023-09-29T23:29:39.612078+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_clean_scrub_stamp":"2023-09-29T23:29:28.522101+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:37:09.181891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620013+0000","last_change":"2023-09-29T23:29:35.099334+0000","last_active":"2023-09-29T23:29:39.620013+0000","last_peered":"2023-09-29T23:29:39.620013+0000","last_clean":"2023-09-29T23:29:39.620013+0000","last_became_active":"2023-09-29T23:29:35.098819+0000","last_became_peered":"2023-09-29T23:29:35.098819+0000","last_unstale":"2023-09-29T23:29:39.620013+0000","last_undegraded":"2023-09-29T23:29:39.620013+0000","last_fullsized":"2023-09-29T23:29:39.620013+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:29:21.699082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617326+0000","last_change":"2023-09-29T23:29:39.272150+0000","last_active":"2023-09-29T23:29:39.617326+0000","last_peered":"2023-09-29T23:29:39.617326+0000","last_clean":"2023-09-29T23:29:39.617326+0000","last_became_active":"2023-09-29T23:29:39.271819+0000","last_became_peered":"2023-09-29T23:29:39.271819+0000","last_unstale":"2023-09-29T23:29:39.617326+0000","last_undegraded":"2023-09-29T23:29:39.617326+0000","last_fullsized":"2023-09-29T23:29:39.617326+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:47.153315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647893+0000","last_change":"2023-09-29T23:29:39.646963+0000","last_active":"2023-09-29T23:29:39.647893+0000","last_peered":"2023-09-29T23:29:39.647893+0000","last_clean":"2023-09-29T23:29:39.647893+0000","last_became_active":"2023-09-29T23:29:39.646666+0000","last_became_peered":"2023-09-29T23:29:39.646666+0000","last_unstale":"2023-09-29T23:29:39.647893+0000","last_undegraded":"2023-09-29T23:29:39.647893+0000","last_fullsized":"2023-09-29T23:29:39.647893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:57:10.275706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617601+0000","last_change":"2023-09-29T23:29:35.098951+0000","last_active":"2023-09-29T23:29:39.617601+0000","last_peered":"2023-09-29T23:29:39.617601+0000","last_clean":"2023-09-29T23:29:39.617601+0000","last_became_active":"2023-09-29T23:29:35.098555+0000","last_became_peered":"2023-09-29T23:29:35.098555+0000","last_unstale":"2023-09-29T23:29:39.617601+0000","last_undegraded":"2023-09-29T23:29:39.617601+0000","last_fullsized":"2023-09-29T23:29:39.617601+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:43:14.246995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597396+0000","last_change":"2023-09-29T23:29:39.260929+0000","last_active":"2023-09-29T23:29:39.597396+0000","last_peered":"2023-09-29T23:29:39.597396+0000","last_clean":"2023-09-29T23:29:39.597396+0000","last_became_active":"2023-09-29T23:29:39.260563+0000","last_became_peered":"2023-09-29T23:29:39.260563+0000","last_unstale":"2023-09-29T23:29:39.597396+0000","last_undegraded":"2023-09-29T23:29:39.597396+0000","last_fullsized":"2023-09-29T23:29:39.597396+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:09:46.708805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.646512+0000","last_change":"2023-09-29T23:29:39.639295+0000","last_active":"2023-09-29T23:29:39.646512+0000","last_peered":"2023-09-29T23:29:39.646512+0000","last_clean":"2023-09-29T23:29:39.646512+0000","last_became_active":"2023-09-29T23:29:39.638232+0000","last_became_peered":"2023-09-29T23:29:39.638232+0000","last_unstale":"2023-09-29T23:29:39.646512+0000","last_undegraded":"2023-09-29T23:29:39.646512+0000","last_fullsized":"2023-09-29T23:29:39.646512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:45:45.006295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.598274+0000","last_change":"2023-09-29T23:29:35.095306+0000","last_active":"2023-09-29T23:29:39.598274+0000","last_peered":"2023-09-29T23:29:39.598274+0000","last_clean":"2023-09-29T23:29:39.598274+0000","last_became_active":"2023-09-29T23:29:35.093309+0000","last_became_peered":"2023-09-29T23:29:35.093309+0000","last_unstale":"2023-09-29T23:29:39.598274+0000","last_undegraded":"2023-09-29T23:29:39.598274+0000","last_fullsized":"2023-09-29T23:29:39.598274+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:10:21.305466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.595615+0000","last_change":"2023-09-29T23:29:39.261319+0000","last_active":"2023-09-29T23:29:39.595615+0000","last_peered":"2023-09-29T23:29:39.595615+0000","last_clean":"2023-09-29T23:29:39.595615+0000","last_became_active":"2023-09-29T23:29:39.260979+0000","last_became_peered":"2023-09-29T23:29:39.260979+0000","last_unstale":"2023-09-29T23:29:39.595615+0000","last_undegraded":"2023-09-29T23:29:39.595615+0000","last_fullsized":"2023-09-29T23:29:39.595615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:58:28.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625251+0000","last_change":"2023-09-29T23:29:39.620458+0000","last_active":"2023-09-29T23:29:39.625251+0000","last_peered":"2023-09-29T23:29:39.625251+0000","last_clean":"2023-09-29T23:29:39.625251+0000","last_became_active":"2023-09-29T23:29:39.620060+0000","last_became_peered":"2023-09-29T23:29:39.620060+0000","last_unstale":"2023-09-29T23:29:39.625251+0000","last_undegraded":"2023-09-29T23:29:39.625251+0000","last_fullsized":"2023-09-29T23:29:39.625251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:49.659581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617399+0000","last_change":"2023-09-29T23:29:35.099643+0000","last_active":"2023-09-29T23:29:39.617399+0000","last_peered":"2023-09-29T23:29:39.617399+0000","last_clean":"2023-09-29T23:29:39.617399+0000","last_became_active":"2023-09-29T23:29:35.099061+0000","last_became_peered":"2023-09-29T23:29:35.099061+0000","last_unstale":"2023-09-29T23:29:39.617399+0000","last_undegraded":"2023-09-29T23:29:39.617399+0000","last_fullsized":"2023-09-29T23:29:39.617399+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:30.050339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.599401+0000","last_change":"2023-09-29T23:29:39.259645+0000","last_active":"2023-09-29T23:29:39.599401+0000","last_peered":"2023-09-29T23:29:39.599401+0000","last_clean":"2023-09-29T23:29:39.599401+0000","last_became_active":"2023-09-29T23:29:39.258895+0000","last_became_peered":"2023-09-29T23:29:39.258895+0000","last_unstale":"2023-09-29T23:29:39.599401+0000","last_undegraded":"2023-09-29T23:29:39.599401+0000","last_fullsized":"2023-09-29T23:29:39.599401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:32:59.650256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609649+0000","last_change":"2023-09-29T23:29:39.608553+0000","last_active":"2023-09-29T23:29:39.609649+0000","last_peered":"2023-09-29T23:29:39.609649+0000","last_clean":"2023-09-29T23:29:39.609649+0000","last_became_active":"2023-09-29T23:29:39.608249+0000","last_became_peered":"2023-09-29T23:29:39.608249+0000","last_unstale":"2023-09-29T23:29:39.609649+0000","last_undegraded":"2023-09-29T23:29:39.609649+0000","last_fullsized":"2023-09-29T23:29:39.609649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:38:28.534318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.599877+0000","last_change":"2023-09-29T23:29:39.259646+0000","last_active":"2023-09-29T23:29:39.599877+0000","last_peered":"2023-09-29T23:29:39.599877+0000","last_clean":"2023-09-29T23:29:39.599877+0000","last_became_active":"2023-09-29T23:29:39.259285+0000","last_became_peered":"2023-09-29T23:29:39.259285+0000","last_unstale":"2023-09-29T23:29:39.599877+0000","last_undegraded":"2023-09-29T23:29:39.599877+0000","last_fullsized":"2023-09-29T23:29:39.599877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:47:29.475878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597541+0000","last_change":"2023-09-29T23:29:35.100113+0000","last_active":"2023-09-29T23:29:39.597541+0000","last_peered":"2023-09-29T23:29:39.597541+0000","last_clean":"2023-09-29T23:29:39.597541+0000","last_became_active":"2023-09-29T23:29:35.099578+0000","last_became_peered":"2023-09-29T23:29:35.099578+0000","last_unstale":"2023-09-29T23:29:39.597541+0000","last_undegraded":"2023-09-29T23:29:39.597541+0000","last_fullsized":"2023-09-29T23:29:39.597541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:57:39.508565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.640050+0000","last_change":"2023-09-29T23:29:39.637174+0000","last_active":"2023-09-29T23:29:39.640050+0000","last_peered":"2023-09-29T23:29:39.640050+0000","last_clean":"2023-09-29T23:29:39.640050+0000","last_became_active":"2023-09-29T23:29:39.636903+0000","last_became_peered":"2023-09-29T23:29:39.636903+0000","last_unstale":"2023-09-29T23:29:39.640050+0000","last_undegraded":"2023-09-29T23:29:39.640050+0000","last_fullsized":"2023-09-29T23:29:39.640050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:04:25.929196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597410+0000","last_change":"2023-09-29T23:29:39.271080+0000","last_active":"2023-09-29T23:29:39.597410+0000","last_peered":"2023-09-29T23:29:39.597410+0000","last_clean":"2023-09-29T23:29:39.597410+0000","last_became_active":"2023-09-29T23:29:39.270673+0000","last_became_peered":"2023-09-29T23:29:39.270673+0000","last_unstale":"2023-09-29T23:29:39.597410+0000","last_undegraded":"2023-09-29T23:29:39.597410+0000","last_fullsized":"2023-09-29T23:29:39.597410+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:28:04.709351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.628891+0000","last_change":"2023-09-29T23:29:35.099325+0000","last_active":"2023-09-29T23:29:39.628891+0000","last_peered":"2023-09-29T23:29:39.628891+0000","last_clean":"2023-09-29T23:29:39.628891+0000","last_became_active":"2023-09-29T23:29:35.098701+0000","last_became_peered":"2023-09-29T23:29:35.098701+0000","last_unstale":"2023-09-29T23:29:39.628891+0000","last_undegraded":"2023-09-29T23:29:39.628891+0000","last_fullsized":"2023-09-29T23:29:39.628891+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:35:07.769754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641024+0000","last_change":"2023-09-29T23:29:39.638042+0000","last_active":"2023-09-29T23:29:39.641024+0000","last_peered":"2023-09-29T23:29:39.641024+0000","last_clean":"2023-09-29T23:29:39.641024+0000","last_became_active":"2023-09-29T23:29:39.637714+0000","last_became_peered":"2023-09-29T23:29:39.637714+0000","last_unstale":"2023-09-29T23:29:39.641024+0000","last_undegraded":"2023-09-29T23:29:39.641024+0000","last_fullsized":"2023-09-29T23:29:39.641024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:47:58.354132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.644775+0000","last_change":"2023-09-29T23:29:39.269267+0000","last_active":"2023-09-29T23:29:39.644775+0000","last_peered":"2023-09-29T23:29:39.644775+0000","last_clean":"2023-09-29T23:29:39.644775+0000","last_became_active":"2023-09-29T23:29:39.268949+0000","last_became_peered":"2023-09-29T23:29:39.268949+0000","last_unstale":"2023-09-29T23:29:39.644775+0000","last_undegraded":"2023-09-29T23:29:39.644775+0000","last_fullsized":"2023-09-29T23:29:39.644775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:07:08.485719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597483+0000","last_change":"2023-09-29T23:29:35.096994+0000","last_active":"2023-09-29T23:29:39.597483+0000","last_peered":"2023-09-29T23:29:39.597483+0000","last_clean":"2023-09-29T23:29:39.597483+0000","last_became_active":"2023-09-29T23:29:35.096568+0000","last_became_peered":"2023-09-29T23:29:35.096568+0000","last_unstale":"2023-09-29T23:29:39.597483+0000","last_undegraded":"2023-09-29T23:29:39.597483+0000","last_fullsized":"2023-09-29T23:29:39.597483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:22:06.698301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645797+0000","last_change":"2023-09-29T23:29:39.638385+0000","last_active":"2023-09-29T23:29:39.645797+0000","last_peered":"2023-09-29T23:29:39.645797+0000","last_clean":"2023-09-29T23:29:39.645797+0000","last_became_active":"2023-09-29T23:29:39.634957+0000","last_became_peered":"2023-09-29T23:29:39.634957+0000","last_unstale":"2023-09-29T23:29:39.645797+0000","last_undegraded":"2023-09-29T23:29:39.645797+0000","last_fullsized":"2023-09-29T23:29:39.645797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:13:11.017795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609533+0000","last_change":"2023-09-29T23:29:39.271004+0000","last_active":"2023-09-29T23:29:39.609533+0000","last_peered":"2023-09-29T23:29:39.609533+0000","last_clean":"2023-09-29T23:29:39.609533+0000","last_became_active":"2023-09-29T23:29:39.270721+0000","last_became_peered":"2023-09-29T23:29:39.270721+0000","last_unstale":"2023-09-29T23:29:39.609533+0000","last_undegraded":"2023-09-29T23:29:39.609533+0000","last_fullsized":"2023-09-29T23:29:39.609533+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:18:38.314574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620608+0000","last_change":"2023-09-29T23:29:35.776424+0000","last_active":"2023-09-29T23:29:39.620608+0000","last_peered":"2023-09-29T23:29:39.620608+0000","last_clean":"2023-09-29T23:29:39.620608+0000","last_became_active":"2023-09-29T23:29:35.775625+0000","last_became_peered":"2023-09-29T23:29:35.775625+0000","last_unstale":"2023-09-29T23:29:39.620608+0000","last_undegraded":"2023-09-29T23:29:39.620608+0000","last_fullsized":"2023-09-29T23:29:39.620608+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:46.078517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.627165+0000","last_change":"2023-09-29T23:29:39.624813+0000","last_active":"2023-09-29T23:29:39.627165+0000","last_peered":"2023-09-29T23:29:39.627165+0000","last_clean":"2023-09-29T23:29:39.627165+0000","last_became_active":"2023-09-29T23:29:39.624521+0000","last_became_peered":"2023-09-29T23:29:39.624521+0000","last_unstale":"2023-09-29T23:29:39.627165+0000","last_undegraded":"2023-09-29T23:29:39.627165+0000","last_fullsized":"2023-09-29T23:29:39.627165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:38:28.534318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.639914+0000","last_change":"2023-09-29T23:29:39.635422+0000","last_active":"2023-09-29T23:29:39.639914+0000","last_peered":"2023-09-29T23:29:39.639914+0000","last_clean":"2023-09-29T23:29:39.639914+0000","last_became_active":"2023-09-29T23:29:39.635139+0000","last_became_peered":"2023-09-29T23:29:39.635139+0000","last_unstale":"2023-09-29T23:29:39.639914+0000","last_undegraded":"2023-09-29T23:29:39.639914+0000","last_fullsized":"2023-09-29T23:29:39.639914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:04:09.469416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"18'6","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647558+0000","last_change":"2023-09-29T23:29:39.271958+0000","last_active":"2023-09-29T23:29:39.647558+0000","last_peered":"2023-09-29T23:29:39.647558+0000","last_clean":"2023-09-29T23:29:39.647558+0000","last_became_active":"2023-09-29T23:29:39.271601+0000","last_became_peered":"2023-09-29T23:29:39.271601+0000","last_unstale":"2023-09-29T23:29:39.647558+0000","last_undegraded":"2023-09-29T23:29:39.647558+0000","last_fullsized":"2023-09-29T23:29:39.647558+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:20:36.278014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637865+0000","last_change":"2023-09-29T23:29:39.633505+0000","last_active":"2023-09-29T23:29:39.637865+0000","last_peered":"2023-09-29T23:29:39.637865+0000","last_clean":"2023-09-29T23:29:39.637865+0000","last_became_active":"2023-09-29T23:29:39.633253+0000","last_became_peered":"2023-09-29T23:29:39.633253+0000","last_unstale":"2023-09-29T23:29:39.637865+0000","last_undegraded":"2023-09-29T23:29:39.637865+0000","last_fullsized":"2023-09-29T23:29:39.637865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:53:25.888059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"18'3","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.633016+0000","last_change":"2023-09-29T23:29:39.268518+0000","last_active":"2023-09-29T23:29:39.633016+0000","last_peered":"2023-09-29T23:29:39.633016+0000","last_clean":"2023-09-29T23:29:39.633016+0000","last_became_active":"2023-09-29T23:29:39.267411+0000","last_became_peered":"2023-09-29T23:29:39.267411+0000","last_unstale":"2023-09-29T23:29:39.633016+0000","last_undegraded":"2023-09-29T23:29:39.633016+0000","last_fullsized":"2023-09-29T23:29:39.633016+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:18:38.314574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.601527+0000","last_change":"2023-09-29T23:29:39.267677+0000","last_active":"2023-09-29T23:29:39.601527+0000","last_peered":"2023-09-29T23:29:39.601527+0000","last_clean":"2023-09-29T23:29:39.601527+0000","last_became_active":"2023-09-29T23:29:39.267374+0000","last_became_peered":"2023-09-29T23:29:39.267374+0000","last_unstale":"2023-09-29T23:29:39.601527+0000","last_undegraded":"2023-09-29T23:29:39.601527+0000","last_fullsized":"2023-09-29T23:29:39.601527+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:23:58.356932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637457+0000","last_change":"2023-09-29T23:29:39.633662+0000","last_active":"2023-09-29T23:29:39.637457+0000","last_peered":"2023-09-29T23:29:39.637457+0000","last_clean":"2023-09-29T23:29:39.637457+0000","last_became_active":"2023-09-29T23:29:39.633327+0000","last_became_peered":"2023-09-29T23:29:39.633327+0000","last_unstale":"2023-09-29T23:29:39.637457+0000","last_undegraded":"2023-09-29T23:29:39.637457+0000","last_fullsized":"2023-09-29T23:29:39.637457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:59:22.894223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.618457+0000","last_change":"2023-09-29T23:29:39.260818+0000","last_active":"2023-09-29T23:29:39.618457+0000","last_peered":"2023-09-29T23:29:39.618457+0000","last_clean":"2023-09-29T23:29:39.618457+0000","last_became_active":"2023-09-29T23:29:39.260529+0000","last_became_peered":"2023-09-29T23:29:39.260529+0000","last_unstale":"2023-09-29T23:29:39.618457+0000","last_undegraded":"2023-09-29T23:29:39.618457+0000","last_fullsized":"2023-09-29T23:29:39.618457+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:47:31.314319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645754+0000","last_change":"2023-09-29T23:29:39.635312+0000","last_active":"2023-09-29T23:29:39.645754+0000","last_peered":"2023-09-29T23:29:39.645754+0000","last_clean":"2023-09-29T23:29:39.645754+0000","last_became_active":"2023-09-29T23:29:39.633112+0000","last_became_peered":"2023-09-29T23:29:39.633112+0000","last_unstale":"2023-09-29T23:29:39.645754+0000","last_undegraded":"2023-09-29T23:29:39.645754+0000","last_fullsized":"2023-09-29T23:29:39.645754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:55:52.762563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645311+0000","last_change":"2023-09-29T23:29:39.635051+0000","last_active":"2023-09-29T23:29:39.645311+0000","last_peered":"2023-09-29T23:29:39.645311+0000","last_clean":"2023-09-29T23:29:39.645311+0000","last_became_active":"2023-09-29T23:29:39.632920+0000","last_became_peered":"2023-09-29T23:29:39.632920+0000","last_unstale":"2023-09-29T23:29:39.645311+0000","last_undegraded":"2023-09-29T23:29:39.645311+0000","last_fullsized":"2023-09-29T23:29:39.645311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:55:37.158884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617319+0000","last_change":"2023-09-29T23:29:39.268725+0000","last_active":"2023-09-29T23:29:39.617319+0000","last_peered":"2023-09-29T23:29:39.617319+0000","last_clean":"2023-09-29T23:29:39.617319+0000","last_became_active":"2023-09-29T23:29:39.268360+0000","last_became_peered":"2023-09-29T23:29:39.268360+0000","last_unstale":"2023-09-29T23:29:39.617319+0000","last_undegraded":"2023-09-29T23:29:39.617319+0000","last_fullsized":"2023-09-29T23:29:39.617319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:23:58.356932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617990+0000","last_change":"2023-09-29T23:29:39.616291+0000","last_active":"2023-09-29T23:29:39.617990+0000","last_peered":"2023-09-29T23:29:39.617990+0000","last_clean":"2023-09-29T23:29:39.617990+0000","last_became_active":"2023-09-29T23:29:39.615966+0000","last_became_peered":"2023-09-29T23:29:39.615966+0000","last_unstale":"2023-09-29T23:29:39.617990+0000","last_undegraded":"2023-09-29T23:29:39.617990+0000","last_fullsized":"2023-09-29T23:29:39.617990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:07:09.489477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.605702+0000","last_change":"2023-09-29T23:29:39.259196+0000","last_active":"2023-09-29T23:29:39.605702+0000","last_peered":"2023-09-29T23:29:39.605702+0000","last_clean":"2023-09-29T23:29:39.605702+0000","last_became_active":"2023-09-29T23:29:39.258785+0000","last_became_peered":"2023-09-29T23:29:39.258785+0000","last_unstale":"2023-09-29T23:29:39.605702+0000","last_undegraded":"2023-09-29T23:29:39.605702+0000","last_fullsized":"2023-09-29T23:29:39.605702+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:23:58.356932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647103+0000","last_change":"2023-09-29T23:29:39.635946+0000","last_active":"2023-09-29T23:29:39.647103+0000","last_peered":"2023-09-29T23:29:39.647103+0000","last_clean":"2023-09-29T23:29:39.647103+0000","last_became_active":"2023-09-29T23:29:39.634129+0000","last_became_peered":"2023-09-29T23:29:39.634129+0000","last_unstale":"2023-09-29T23:29:39.647103+0000","last_undegraded":"2023-09-29T23:29:39.647103+0000","last_fullsized":"2023-09-29T23:29:39.647103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:31:36.612565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629612+0000","last_change":"2023-09-29T23:29:39.252290+0000","last_active":"2023-09-29T23:29:39.629612+0000","last_peered":"2023-09-29T23:29:39.629612+0000","last_clean":"2023-09-29T23:29:39.629612+0000","last_became_active":"2023-09-29T23:29:39.246835+0000","last_became_peered":"2023-09-29T23:29:39.246835+0000","last_unstale":"2023-09-29T23:29:39.629612+0000","last_undegraded":"2023-09-29T23:29:39.629612+0000","last_fullsized":"2023-09-29T23:29:39.629612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:09:46.708805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609639+0000","last_change":"2023-09-29T23:29:39.608519+0000","last_active":"2023-09-29T23:29:39.609639+0000","last_peered":"2023-09-29T23:29:39.609639+0000","last_clean":"2023-09-29T23:29:39.609639+0000","last_became_active":"2023-09-29T23:29:39.608334+0000","last_became_peered":"2023-09-29T23:29:39.608334+0000","last_unstale":"2023-09-29T23:29:39.609639+0000","last_undegraded":"2023-09-29T23:29:39.609639+0000","last_fullsized":"2023-09-29T23:29:39.609639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:31.379677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.632767+0000","last_change":"2023-09-29T23:29:39.270477+0000","last_active":"2023-09-29T23:29:39.632767+0000","last_peered":"2023-09-29T23:29:39.632767+0000","last_clean":"2023-09-29T23:29:39.632767+0000","last_became_active":"2023-09-29T23:29:39.270178+0000","last_became_peered":"2023-09-29T23:29:39.270178+0000","last_unstale":"2023-09-29T23:29:39.632767+0000","last_undegraded":"2023-09-29T23:29:39.632767+0000","last_fullsized":"2023-09-29T23:29:39.632767+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:05:51.179867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.608417+0000","last_change":"2023-09-29T23:29:39.605080+0000","last_active":"2023-09-29T23:29:39.608417+0000","last_peered":"2023-09-29T23:29:39.608417+0000","last_clean":"2023-09-29T23:29:39.608417+0000","last_became_active":"2023-09-29T23:29:39.604701+0000","last_became_peered":"2023-09-29T23:29:39.604701+0000","last_unstale":"2023-09-29T23:29:39.608417+0000","last_undegraded":"2023-09-29T23:29:39.608417+0000","last_fullsized":"2023-09-29T23:29:39.608417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:49.659581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645866+0000","last_change":"2023-09-29T23:29:39.259176+0000","last_active":"2023-09-29T23:29:39.645866+0000","last_peered":"2023-09-29T23:29:39.645866+0000","last_clean":"2023-09-29T23:29:39.645866+0000","last_became_active":"2023-09-29T23:29:39.258811+0000","last_became_peered":"2023-09-29T23:29:39.258811+0000","last_unstale":"2023-09-29T23:29:39.645866+0000","last_undegraded":"2023-09-29T23:29:39.645866+0000","last_fullsized":"2023-09-29T23:29:39.645866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:38:39.409933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.608584+0000","last_change":"2023-09-29T23:29:39.606077+0000","last_active":"2023-09-29T23:29:39.608584+0000","last_peered":"2023-09-29T23:29:39.608584+0000","last_clean":"2023-09-29T23:29:39.608584+0000","last_became_active":"2023-09-29T23:29:39.605747+0000","last_became_peered":"2023-09-29T23:29:39.605747+0000","last_unstale":"2023-09-29T23:29:39.608584+0000","last_undegraded":"2023-09-29T23:29:39.608584+0000","last_fullsized":"2023-09-29T23:29:39.608584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:07:09.489477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.644653+0000","last_change":"2023-09-29T23:29:39.273703+0000","last_active":"2023-09-29T23:29:39.644653+0000","last_peered":"2023-09-29T23:29:39.644653+0000","last_clean":"2023-09-29T23:29:39.644653+0000","last_became_active":"2023-09-29T23:29:39.273385+0000","last_became_peered":"2023-09-29T23:29:39.273385+0000","last_unstale":"2023-09-29T23:29:39.644653+0000","last_undegraded":"2023-09-29T23:29:39.644653+0000","last_fullsized":"2023-09-29T23:29:39.644653+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:04:08.465658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645244+0000","last_change":"2023-09-29T23:29:39.634607+0000","last_active":"2023-09-29T23:29:39.645244+0000","last_peered":"2023-09-29T23:29:39.645244+0000","last_clean":"2023-09-29T23:29:39.645244+0000","last_became_active":"2023-09-29T23:29:39.633666+0000","last_became_peered":"2023-09-29T23:29:39.633666+0000","last_unstale":"2023-09-29T23:29:39.645244+0000","last_undegraded":"2023-09-29T23:29:39.645244+0000","last_fullsized":"2023-09-29T23:29:39.645244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:19:23.819855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609036+0000","last_change":"2023-09-29T23:29:39.271329+0000","last_active":"2023-09-29T23:29:39.609036+0000","last_peered":"2023-09-29T23:29:39.609036+0000","last_clean":"2023-09-29T23:29:39.609036+0000","last_became_active":"2023-09-29T23:29:39.270902+0000","last_became_peered":"2023-09-29T23:29:39.270902+0000","last_unstale":"2023-09-29T23:29:39.609036+0000","last_undegraded":"2023-09-29T23:29:39.609036+0000","last_fullsized":"2023-09-29T23:29:39.609036+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:43:32.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609490+0000","last_change":"2023-09-29T23:29:39.605447+0000","last_active":"2023-09-29T23:29:39.609490+0000","last_peered":"2023-09-29T23:29:39.609490+0000","last_clean":"2023-09-29T23:29:39.609490+0000","last_became_active":"2023-09-29T23:29:39.604800+0000","last_became_peered":"2023-09-29T23:29:39.604800+0000","last_unstale":"2023-09-29T23:29:39.609490+0000","last_undegraded":"2023-09-29T23:29:39.609490+0000","last_fullsized":"2023-09-29T23:29:39.609490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:58:35.776573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"18'4","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.631433+0000","last_change":"2023-09-29T23:29:39.251909+0000","last_active":"2023-09-29T23:29:39.631433+0000","last_peered":"2023-09-29T23:29:39.631433+0000","last_clean":"2023-09-29T23:29:39.631433+0000","last_became_active":"2023-09-29T23:29:39.251464+0000","last_became_peered":"2023-09-29T23:29:39.251464+0000","last_unstale":"2023-09-29T23:29:39.631433+0000","last_undegraded":"2023-09-29T23:29:39.631433+0000","last_fullsized":"2023-09-29T23:29:39.631433+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:58:28.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705670,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12684,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":12480,"kb_avail":94359156,"statfs":{"total":96636764160,"available":96623775744,"internally_reserved":0,"allocated":200704,"data_stored":62103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12779520},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13132,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358708,"statfs":{"total":96636764160,"available":96623316992,"internally_reserved":0,"allocated":266240,"data_stored":652423,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79400000000000004}]}]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13120,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358720,"statfs":{"total":96636764160,"available":96623329280,"internally_reserved":0,"allocated":253952,"data_stored":651387,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8440,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363400,"statfs":{"total":96636764160,"available":96628121600,"internally_reserved":0,"allocated":180224,"data_stored":60927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T23:29:47.242 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-29T23:29:47.243 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-29T23:29:47.243 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-29T23:29:47.243 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-29T23:29:47.243 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-09-29T23:29:47.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d3fff700 1 Processor -- start 2023-09-29T23:29:47.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d3fff700 1 -- start start 2023-09-29T23:29:47.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d3fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f88d4066ec0 0x7f88d40a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4148590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d3fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d40a16b0 con 0x7f88d409f830 2023-09-29T23:29:47.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d3fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d4067290 con 0x7f88d4066ec0 2023-09-29T23:29:47.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d3fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40a0510 con 0x7f88d409fc70 2023-09-29T23:29:47.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d37fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d37fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60344/0 (socket says 172.21.15.143:60344) 2023-09-29T23:29:47.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d37fe700 1 -- 172.21.15.143:0/2056881676 learned_addr learned my addr 172.21.15.143:0/2056881676 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:47.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d2ffd700 1 --2- 172.21.15.143:0/2056881676 >> v2:172.21.15.156:3300/0 conn(0x7f88d4066ec0 0x7f88d40a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d37fe700 1 -- 172.21.15.143:0/2056881676 >> v2:172.21.15.156:3300/0 conn(0x7f88d4066ec0 msgr2=0x7f88d40a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:47.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d37fe700 1 --2- 172.21.15.143:0/2056881676 >> v2:172.21.15.156:3300/0 conn(0x7f88d4066ec0 0x7f88d40a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d37fe700 1 -- 172.21.15.143:0/2056881676 >> v1:172.21.15.156:6789/0 conn(0x7f88d409f830 legacy=0x7f88d413d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d37fe700 1 -- 172.21.15.143:0/2056881676 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d406e400 con 0x7f88d409fc70 2023-09-29T23:29:47.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d37fe700 1 --2- 172.21.15.143:0/2056881676 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4148590 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009f20 tx=0x7f88cc009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e3ab37446ceeb06 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d1ffb700 1 -- 172.21.15.143:0/2056881676 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00e070 con 0x7f88d409fc70 2023-09-29T23:29:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d1ffb700 1 -- 172.21.15.143:0/2056881676 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc00ba80 con 0x7f88d409fc70 2023-09-29T23:29:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.344+0000 7f88d1ffb700 1 -- 172.21.15.143:0/2056881676 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0092e0 con 0x7f88d409fc70 2023-09-29T23:29:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.348+0000 7f88d3fff700 1 -- 172.21.15.143:0/2056881676 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 msgr2=0x7f88d4148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.348+0000 7f88d3fff700 1 --2- 172.21.15.143:0/2056881676 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4148590 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f88cc009f20 tx=0x7f88cc009840 comp rx=0 tx=0).stop 2023-09-29T23:29:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.348+0000 7f88d3fff700 1 -- 172.21.15.143:0/2056881676 shutdown_connections 2023-09-29T23:29:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.348+0000 7f88d3fff700 1 --2- 172.21.15.143:0/2056881676 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4148590 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.348+0000 7f88d3fff700 1 --2- 172.21.15.143:0/2056881676 >> v2:172.21.15.156:3300/0 conn(0x7f88d4066ec0 0x7f88d40a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.348+0000 7f88d3fff700 1 -- 172.21.15.143:0/2056881676 >> 172.21.15.143:0/2056881676 conn(0x7f88d406f200 msgr2=0x7f88d4069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.348+0000 7f88d3fff700 1 -- 172.21.15.143:0/2056881676 shutdown_connections 2023-09-29T23:29:47.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.348+0000 7f88d3fff700 1 -- 172.21.15.143:0/2056881676 wait complete. 2023-09-29T23:29:47.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.348+0000 7f88d3fff700 1 Processor -- start 2023-09-29T23:29:47.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.348+0000 7f88d3fff700 1 -- start start 2023-09-29T23:29:47.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88d4066ec0 0x7f88d40a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d409f830 0x7f88d4140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4137050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d3fff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d4097060 con 0x7f88d409f830 2023-09-29T23:29:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d3fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d4097320 con 0x7f88d4066ec0 2023-09-29T23:29:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d3fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d4141ff0 con 0x7f88d409fc70 2023-09-29T23:29:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d37fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4137050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d27fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d409f830 0x7f88d4140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d37fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4137050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60356/0 (socket says 172.21.15.143:60356) 2023-09-29T23:29:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d27fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d409f830 0x7f88d4140e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34468/0 (socket says 172.21.15.143:34468) 2023-09-29T23:29:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d37fe700 1 -- 172.21.15.143:0/4150152678 learned_addr learned my addr 172.21.15.143:0/4150152678 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d2ffd700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88d4066ec0 0x7f88d40a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d37fe700 1 -- 172.21.15.143:0/4150152678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88d4066ec0 msgr2=0x7f88d40a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d37fe700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88d4066ec0 0x7f88d40a4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d37fe700 1 -- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d409f830 msgr2=0x7f88d4140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d37fe700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d409f830 0x7f88d4140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d37fe700 1 -- 172.21.15.143:0/4150152678 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc009ae0 con 0x7f88d409fc70 2023-09-29T23:29:47.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d27fc700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d409f830 0x7f88d4140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:47.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d37fe700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4137050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00b6c0 tx=0x7f88cc018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:47.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88baffd700 1 -- 172.21.15.143:0/4150152678 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc0032f0 con 0x7f88d409fc70 2023-09-29T23:29:47.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88baffd700 1 -- 172.21.15.143:0/4150152678 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc007cd0 con 0x7f88d409fc70 2023-09-29T23:29:47.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88baffd700 1 -- 172.21.15.143:0/4150152678 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc01d630 con 0x7f88d409fc70 2023-09-29T23:29:47.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d3fff700 1 -- 172.21.15.143:0/4150152678 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d40962a0 con 0x7f88d409fc70 2023-09-29T23:29:47.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d3fff700 1 -- 172.21.15.143:0/4150152678 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d41404a0 con 0x7f88d409fc70 2023-09-29T23:29:47.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.352+0000 7f88d3fff700 1 -- 172.21.15.143:0/4150152678 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d4095fa0 con 0x7f88d409fc70 2023-09-29T23:29:47.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.356+0000 7f88baffd700 1 -- 172.21.15.143:0/4150152678 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f88cc003040 con 0x7f88d409fc70 2023-09-29T23:29:47.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.356+0000 7f88baffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.356+0000 7f88baffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.356+0000 7f88baffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.356+0000 7f88baffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.356+0000 7f88baffd700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f88c008e8e0 0x7f88c0090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.356+0000 7f88d2ffd700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f88c008e8e0 0x7f88c0090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.356+0000 7f88baffd700 1 -- 172.21.15.143:0/4150152678 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f88cc0b61e0 con 0x7f88d409fc70 2023-09-29T23:29:47.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.356+0000 7f88d2ffd700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f88c008e8e0 0x7f88c0090d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f88c8006fd0 tx=0x7f88c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:47.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.364+0000 7f88baffd700 1 -- 172.21.15.143:0/4150152678 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f88cc086ca0 con 0x7f88d409fc70 2023-09-29T23:29:47.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.564+0000 7f88d3fff700 1 -- 172.21.15.143:0/4150152678 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f88d405ecb0 con 0x7f88d409fc70 2023-09-29T23:29:47.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.576+0000 7f88baffd700 1 -- 172.21.15.143:0/4150152678 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+230617 (secure 0 0 0) 0x7f88cc08a2c0 con 0x7f88d409fc70 2023-09-29T23:29:47.581 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:47.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 10 mgrc shutdown 2023-09-29T23:29:47.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 -- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f88c008e8e0 msgr2=0x7f88c0090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f88c008e8e0 0x7f88c0090d90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f88c8006fd0 tx=0x7f88c8008040 comp rx=0 tx=0).stop 2023-09-29T23:29:47.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 -- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 msgr2=0x7f88d4137050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4137050 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f88cc00b6c0 tx=0x7f88cc018000 comp rx=0 tx=0).stop 2023-09-29T23:29:47.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 -- 172.21.15.143:0/4150152678 shutdown_connections 2023-09-29T23:29:47.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f88c008e8e0 0x7f88c0090d90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88d409fc70 0x7f88d4137050 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d409f830 0x7f88d4140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 --2- 172.21.15.143:0/4150152678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88d4066ec0 0x7f88d40a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 -- 172.21.15.143:0/4150152678 >> 172.21.15.143:0/4150152678 conn(0x7f88d406f200 msgr2=0x7f88d406fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:47.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 -- 172.21.15.143:0/4150152678 shutdown_connections 2023-09-29T23:29:47.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.580+0000 7f88d3fff700 1 -- 172.21.15.143:0/4150152678 wait complete. 2023-09-29T23:29:47.595 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":5,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":27602},{"type":"v1","addr":"172.21.15.143:6809","nonce":27602}]},"active_addr":"172.21.15.143:6809/27602","active_change":"2023-09-29T23:29:22.447754+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4107,"name":"z","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":4114,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":3540896560}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":2311379769}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":2847257452}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.143:0","nonce":1913586509}]}]} 2023-09-29T23:29:47.601 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-29T23:29:47.601 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-29T23:29:47.602 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T23:29:47.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.704+0000 7f5cb3fff700 1 Processor -- start 2023-09-29T23:29:47.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.704+0000 7f5cb3fff700 1 -- start start 2023-09-29T23:29:47.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb3fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5cb4066ec0 0x7f5cb40a8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb3fff700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb40a86d0 con 0x7f5cb413f8e0 2023-09-29T23:29:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb3fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb40a4950 con 0x7f5cb40a5130 2023-09-29T23:29:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb3fff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb4067290 con 0x7f5cb4066ec0 2023-09-29T23:29:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb2ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5cb4066ec0 0x7f5cb40a8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb2ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f5cb4066ec0 0x7f5cb40a8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34476/0 (socket says 172.21.15.143:34476) 2023-09-29T23:29:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb2ffd700 1 -- 172.21.15.143:0/4256767880 learned_addr learned my addr 172.21.15.143:0/4256767880 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb37fe700 1 --1- 172.21.15.143:0/4256767880 >> v1:172.21.15.143:6789/0 conn(0x7f5cb413f8e0 0x7f5cb414ab90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60648/0 (socket says 172.21.15.143:60648) 2023-09-29T23:29:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb2ffd700 1 -- 172.21.15.143:0/4256767880 >> v1:172.21.15.156:6789/0 conn(0x7f5cb40a5130 legacy=0x7f5cb40a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb2ffd700 1 -- 172.21.15.143:0/4256767880 >> v1:172.21.15.143:6789/0 conn(0x7f5cb413f8e0 legacy=0x7f5cb414ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb2ffd700 1 -- 172.21.15.143:0/4256767880 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb40a9060 con 0x7f5cb4066ec0 2023-09-29T23:29:47.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb2ffd700 1 --2- 172.21.15.143:0/4256767880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb4066ec0 0x7f5cb40a8b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40099b0 tx=0x7f5ca4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8403e6dc287720c server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:47.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb1ffb700 1 -- 172.21.15.143:0/4256767880 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400b860 con 0x7f5cb4066ec0 2023-09-29T23:29:47.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb1ffb700 1 -- 172.21.15.143:0/4256767880 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca400b9c0 con 0x7f5cb4066ec0 2023-09-29T23:29:47.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb1ffb700 1 -- 172.21.15.143:0/4256767880 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400bd60 con 0x7f5cb4066ec0 2023-09-29T23:29:47.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb3fff700 1 -- 172.21.15.143:0/4256767880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb4066ec0 msgr2=0x7f5cb40a8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb3fff700 1 --2- 172.21.15.143:0/4256767880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb4066ec0 0x7f5cb40a8b20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40099b0 tx=0x7f5ca4009250 comp rx=0 tx=0).stop 2023-09-29T23:29:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb3fff700 1 -- 172.21.15.143:0/4256767880 shutdown_connections 2023-09-29T23:29:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb3fff700 1 --2- 172.21.15.143:0/4256767880 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb4066ec0 0x7f5cb40a8b20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb3fff700 1 -- 172.21.15.143:0/4256767880 >> 172.21.15.143:0/4256767880 conn(0x7f5cb406ee30 msgr2=0x7f5cb4069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb3fff700 1 -- 172.21.15.143:0/4256767880 shutdown_connections 2023-09-29T23:29:47.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.708+0000 7f5cb3fff700 1 -- 172.21.15.143:0/4256767880 wait complete. 2023-09-29T23:29:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb3fff700 1 Processor -- start 2023-09-29T23:29:47.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb3fff700 1 -- start start 2023-09-29T23:29:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb3fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cb4066ec0 0x7f5cb4142490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb40a5130 0x7f5cb406aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb3fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb413f8e0 0x7f5cb406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb3fff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4142e60 con 0x7f5cb413f8e0 2023-09-29T23:29:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb3fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb4143120 con 0x7f5cb40a5130 2023-09-29T23:29:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb3fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb41433e0 con 0x7f5cb4066ec0 2023-09-29T23:29:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb2ffd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cb4066ec0 0x7f5cb4142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb2ffd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cb4066ec0 0x7f5cb4142490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60372/0 (socket says 172.21.15.143:60372) 2023-09-29T23:29:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb2ffd700 1 -- 172.21.15.143:0/447064141 learned_addr learned my addr 172.21.15.143:0/447064141 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb37fe700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb413f8e0 0x7f5cb406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb27fc700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb40a5130 0x7f5cb406aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb2ffd700 1 -- 172.21.15.143:0/447064141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb40a5130 msgr2=0x7f5cb406aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb2ffd700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb40a5130 0x7f5cb406aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb2ffd700 1 -- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb413f8e0 msgr2=0x7f5cb406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb2ffd700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb413f8e0 0x7f5cb406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.712+0000 7f5cb2ffd700 1 -- 172.21.15.143:0/447064141 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca40094f0 con 0x7f5cb4066ec0 2023-09-29T23:29:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.716+0000 7f5cb2ffd700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cb4066ec0 0x7f5cb4142490 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009290 tx=0x7f5ca4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.716+0000 7f5c9affd700 1 -- 172.21.15.143:0/447064141 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca401a700 con 0x7f5cb4066ec0 2023-09-29T23:29:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.716+0000 7f5c9affd700 1 -- 172.21.15.143:0/447064141 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca401a860 con 0x7f5cb4066ec0 2023-09-29T23:29:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.716+0000 7f5c9affd700 1 -- 172.21.15.143:0/447064141 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca401ac20 con 0x7f5cb4066ec0 2023-09-29T23:29:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.716+0000 7f5cb3fff700 1 -- 172.21.15.143:0/447064141 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb4141b80 con 0x7f5cb4066ec0 2023-09-29T23:29:47.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.716+0000 7f5cb3fff700 1 -- 172.21.15.143:0/447064141 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb4141650 con 0x7f5cb4066ec0 2023-09-29T23:29:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.716+0000 7f5cb3fff700 1 -- 172.21.15.143:0/447064141 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb405ecb0 con 0x7f5cb4066ec0 2023-09-29T23:29:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.716+0000 7f5c9affd700 1 -- 172.21.15.143:0/447064141 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f5ca40253f0 con 0x7f5cb4066ec0 2023-09-29T23:29:47.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.716+0000 7f5c9affd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:47.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.720+0000 7f5c9affd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:47.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.720+0000 7f5c9affd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:47.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.720+0000 7f5c9affd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:47.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.720+0000 7f5c9affd700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5ca008e8b0 0x7f5ca0090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:47.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.720+0000 7f5c9affd700 1 -- 172.21.15.143:0/447064141 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f5ca40b6570 con 0x7f5cb4066ec0 2023-09-29T23:29:47.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.720+0000 7f5cb27fc700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5ca008e8b0 0x7f5ca0090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:47.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.720+0000 7f5cb27fc700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5ca008e8b0 0x7f5ca0090d60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40a3f70 tx=0x7f5ca8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:47.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.728+0000 7f5c9affd700 1 -- 172.21.15.143:0/447064141 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5ca4087030 con 0x7f5cb4066ec0 2023-09-29T23:29:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.884+0000 7f5cb3fff700 1 -- 172.21.15.143:0/447064141 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5cb404b5c0 con 0x7f5cb4066ec0 2023-09-29T23:29:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5c9affd700 1 -- 172.21.15.143:0/447064141 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12206 (secure 0 0 0) 0x7f5ca408a650 con 0x7f5cb4066ec0 2023-09-29T23:29:47.890 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:47.890 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:29:40.590094+0000","last_up_change":"2023-09-29T23:29:27.340059+0000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T23:29:28.520729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T23:29:33.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-29T23:29:37.927418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-29T23:29:38.594861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":27741},{"type":"v1","addr":"172.21.15.143:6801","nonce":27741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":27741},{"type":"v1","addr":"172.21.15.143:6803","nonce":27741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":27741},{"type":"v1","addr":"172.21.15.143:6807","nonce":27741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":27741},{"type":"v1","addr":"172.21.15.143:6805","nonce":27741}]},"public_addr":"172.21.15.143:6801/27741","cluster_addr":"172.21.15.143:6803/27741","heartbeat_back_addr":"172.21.15.143:6807/27741","heartbeat_front_addr":"172.21.15.143:6805/27741","state":["exists","up"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":30036},{"type":"v1","addr":"172.21.15.156:6817","nonce":30036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":30036},{"type":"v1","addr":"172.21.15.156:6819","nonce":30036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":30036},{"type":"v1","addr":"172.21.15.156:6823","nonce":30036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":30036},{"type":"v1","addr":"172.21.15.156:6821","nonce":30036}]},"public_addr":"172.21.15.156:6817/30036","cluster_addr":"172.21.15.156:6819/30036","heartbeat_back_addr":"172.21.15.156:6823/30036","heartbeat_front_addr":"172.21.15.156:6821/30036","state":["exists","up"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":30035},{"type":"v1","addr":"172.21.15.156:6809","nonce":30035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":30035},{"type":"v1","addr":"172.21.15.156:6811","nonce":30035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":30035},{"type":"v1","addr":"172.21.15.156:6815","nonce":30035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":30035},{"type":"v1","addr":"172.21.15.156:6813","nonce":30035}]},"public_addr":"172.21.15.156:6809/30035","cluster_addr":"172.21.15.156:6811/30035","heartbeat_back_addr":"172.21.15.156:6815/30035","heartbeat_front_addr":"172.21.15.156:6813/30035","state":["exists","up"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":30037},{"type":"v1","addr":"172.21.15.156:6801","nonce":30037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":30037},{"type":"v1","addr":"172.21.15.156:6803","nonce":30037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":30037},{"type":"v1","addr":"172.21.15.156:6807","nonce":30037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":30037},{"type":"v1","addr":"172.21.15.156:6805","nonce":30037}]},"public_addr":"172.21.15.156:6801/30037","cluster_addr":"172.21.15.156:6803/30037","heartbeat_back_addr":"172.21.15.156:6807/30037","heartbeat_front_addr":"172.21.15.156:6805/30037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.781977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.943955+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.806513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.624301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:29:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 10 mgrc shutdown 2023-09-29T23:29:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 -- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5ca008e8b0 msgr2=0x7f5ca0090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5ca008e8b0 0x7f5ca0090d60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40a3f70 tx=0x7f5ca8009040 comp rx=0 tx=0).stop 2023-09-29T23:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 -- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cb4066ec0 msgr2=0x7f5cb4142490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cb4066ec0 0x7f5cb4142490 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ca4009290 tx=0x7f5ca4017000 comp rx=0 tx=0).stop 2023-09-29T23:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 -- 172.21.15.143:0/447064141 shutdown_connections 2023-09-29T23:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5ca008e8b0 0x7f5ca0090d60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cb413f8e0 0x7f5cb406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cb40a5130 0x7f5cb406aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 --2- 172.21.15.143:0/447064141 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5cb4066ec0 0x7f5cb4142490 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 -- 172.21.15.143:0/447064141 >> 172.21.15.143:0/447064141 conn(0x7f5cb406ee30 msgr2=0x7f5cb413cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 -- 172.21.15.143:0/447064141 shutdown_connections 2023-09-29T23:29:47.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:47.888+0000 7f5cb3fff700 1 -- 172.21.15.143:0/447064141 wait complete. 2023-09-29T23:29:47.904 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-29T23:29:47.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T23:29:48.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.008+0000 7fade2c78700 1 Processor -- start 2023-09-29T23:29:48.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.008+0000 7fade2c78700 1 -- start start 2023-09-29T23:29:48.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faddc0b30b0 0x7faddc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faddc1480a0 con 0x7faddc147c30 2023-09-29T23:29:48.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faddc06b230 con 0x7faddc06acd0 2023-09-29T23:29:48.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faddc06b4f0 con 0x7faddc0b30b0 2023-09-29T23:29:48.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade0a14700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faddc0b30b0 0x7faddc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7faddbfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7faddc147c30 0x7faddc148530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60658/0 (socket says 172.21.15.143:60658) 2023-09-29T23:29:48.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7faddbfff700 1 -- 172.21.15.143:0/3315515883 learned_addr learned my addr 172.21.15.143:0/3315515883 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:48.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade0a14700 1 -- 172.21.15.143:0/3315515883 >> v1:172.21.15.156:6789/0 conn(0x7faddc06acd0 legacy=0x7faddc13a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade0a14700 1 -- 172.21.15.143:0/3315515883 >> v1:172.21.15.143:6789/0 conn(0x7faddc147c30 legacy=0x7faddc148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7faddb7fe700 1 -- 172.21.15.143:0/3315515883 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196975709 0 0) 0x7faddc1480a0 con 0x7faddc147c30 2023-09-29T23:29:48.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade0a14700 1 -- 172.21.15.143:0/3315515883 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faddc0b3480 con 0x7faddc0b30b0 2023-09-29T23:29:48.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade0a14700 1 --2- 172.21.15.143:0/3315515883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faddc0b30b0 0x7faddc06d870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009e60 tx=0x7fadd4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fe2a76433651a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7faddb7fe700 1 -- 172.21.15.143:0/3315515883 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd400e070 con 0x7faddc0b30b0 2023-09-29T23:29:48.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7faddb7fe700 1 -- 172.21.15.143:0/3315515883 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd400bc30 con 0x7faddc0b30b0 2023-09-29T23:29:48.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7faddb7fe700 1 -- 172.21.15.143:0/3315515883 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd4012430 con 0x7faddc0b30b0 2023-09-29T23:29:48.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 -- 172.21.15.143:0/3315515883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faddc0b30b0 msgr2=0x7faddc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 --2- 172.21.15.143:0/3315515883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faddc0b30b0 0x7faddc06d870 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009e60 tx=0x7fadd4009780 comp rx=0 tx=0).stop 2023-09-29T23:29:48.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 -- 172.21.15.143:0/3315515883 shutdown_connections 2023-09-29T23:29:48.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 --2- 172.21.15.143:0/3315515883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faddc0b30b0 0x7faddc06d870 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.017 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 -- 172.21.15.143:0/3315515883 >> 172.21.15.143:0/3315515883 conn(0x7faddc067dc0 msgr2=0x7faddc0b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 -- 172.21.15.143:0/3315515883 shutdown_connections 2023-09-29T23:29:48.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 -- 172.21.15.143:0/3315515883 wait complete. 2023-09-29T23:29:48.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.012+0000 7fade2c78700 1 Processor -- start 2023-09-29T23:29:48.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade2c78700 1 -- start start 2023-09-29T23:29:48.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade2c78700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc06acd0 0x7faddc138bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade2c78700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faddc0b30b0 0x7faddc1366d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade2c78700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faddc147c30 0x7faddc0a8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade2c78700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc0aac90 con 0x7faddc0b30b0 2023-09-29T23:29:48.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade2c78700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faddc0af640 con 0x7faddc06acd0 2023-09-29T23:29:48.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade2c78700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faddc1371e0 con 0x7faddc147c30 2023-09-29T23:29:48.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7faddbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faddc0b30b0 0x7faddc1366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade1215700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faddc147c30 0x7faddc0a8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7faddbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faddc0b30b0 0x7faddc1366d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34498/0 (socket says 172.21.15.143:34498) 2023-09-29T23:29:48.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade0a14700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc06acd0 0x7faddc138bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7faddbfff700 1 -- 172.21.15.143:0/4138802144 learned_addr learned my addr 172.21.15.143:0/4138802144 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:48.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade1215700 1 -- 172.21.15.143:0/4138802144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc06acd0 msgr2=0x7faddc138bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade1215700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc06acd0 0x7faddc138bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade1215700 1 -- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faddc0b30b0 msgr2=0x7faddc1366d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade1215700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faddc0b30b0 0x7faddc1366d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade1215700 1 -- 172.21.15.143:0/4138802144 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd4009a20 con 0x7faddc147c30 2023-09-29T23:29:48.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade1215700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faddc147c30 0x7faddc0a8bb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00c4e0 tx=0x7fadcc007e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fadd8ff9700 1 -- 172.21.15.143:0/4138802144 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc00ce50 con 0x7faddc147c30 2023-09-29T23:29:48.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fadd8ff9700 1 -- 172.21.15.143:0/4138802144 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc011590 con 0x7faddc147c30 2023-09-29T23:29:48.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade2c78700 1 -- 172.21.15.143:0/4138802144 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faddc1374a0 con 0x7faddc147c30 2023-09-29T23:29:48.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fadd8ff9700 1 -- 172.21.15.143:0/4138802144 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc011950 con 0x7faddc147c30 2023-09-29T23:29:48.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade2c78700 1 -- 172.21.15.143:0/4138802144 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faddc139a90 con 0x7faddc147c30 2023-09-29T23:29:48.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.016+0000 7fade2c78700 1 -- 172.21.15.143:0/4138802144 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faddc05ecb0 con 0x7faddc147c30 2023-09-29T23:29:48.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.020+0000 7fadd8ff9700 1 -- 172.21.15.143:0/4138802144 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fadcc011d00 con 0x7faddc147c30 2023-09-29T23:29:48.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.020+0000 7fadd8ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:48.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.020+0000 7fadd8ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:48.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.020+0000 7fadd8ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.020+0000 7fadd8ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.020+0000 7fadd8ff9700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fadc808e830 0x7fadc8090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.020+0000 7fade0a14700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fadc808e830 0x7fadc8090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.020+0000 7fadd8ff9700 1 -- 172.21.15.143:0/4138802144 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7fadcc0b4020 con 0x7faddc147c30 2023-09-29T23:29:48.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.020+0000 7fade0a14700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fadc808e830 0x7fadc8090ce0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009e60 tx=0x7fadd4019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.028+0000 7fadd8ff9700 1 -- 172.21.15.143:0/4138802144 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fadcc084a50 con 0x7faddc147c30 2023-09-29T23:29:48.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.188+0000 7fade2c78700 1 -- 172.21.15.143:0/4138802144 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faddc04b5c0 con 0x7faddc147c30 2023-09-29T23:29:48.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.188+0000 7fadd8ff9700 1 -- 172.21.15.143:0/4138802144 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12206 (secure 0 0 0) 0x7fadcc017030 con 0x7faddc147c30 2023-09-29T23:29:48.191 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:48.191 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":19,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:29:40.590094+0000","last_up_change":"2023-09-29T23:29:27.340059+0000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T23:29:28.520729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T23:29:33.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-29T23:29:37.927418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-29T23:29:38.594861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":27741},{"type":"v1","addr":"172.21.15.143:6801","nonce":27741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":27741},{"type":"v1","addr":"172.21.15.143:6803","nonce":27741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":27741},{"type":"v1","addr":"172.21.15.143:6807","nonce":27741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":27741},{"type":"v1","addr":"172.21.15.143:6805","nonce":27741}]},"public_addr":"172.21.15.143:6801/27741","cluster_addr":"172.21.15.143:6803/27741","heartbeat_back_addr":"172.21.15.143:6807/27741","heartbeat_front_addr":"172.21.15.143:6805/27741","state":["exists","up"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":30036},{"type":"v1","addr":"172.21.15.156:6817","nonce":30036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":30036},{"type":"v1","addr":"172.21.15.156:6819","nonce":30036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":30036},{"type":"v1","addr":"172.21.15.156:6823","nonce":30036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":30036},{"type":"v1","addr":"172.21.15.156:6821","nonce":30036}]},"public_addr":"172.21.15.156:6817/30036","cluster_addr":"172.21.15.156:6819/30036","heartbeat_back_addr":"172.21.15.156:6823/30036","heartbeat_front_addr":"172.21.15.156:6821/30036","state":["exists","up"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":30035},{"type":"v1","addr":"172.21.15.156:6809","nonce":30035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":30035},{"type":"v1","addr":"172.21.15.156:6811","nonce":30035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":30035},{"type":"v1","addr":"172.21.15.156:6815","nonce":30035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":30035},{"type":"v1","addr":"172.21.15.156:6813","nonce":30035}]},"public_addr":"172.21.15.156:6809/30035","cluster_addr":"172.21.15.156:6811/30035","heartbeat_back_addr":"172.21.15.156:6815/30035","heartbeat_front_addr":"172.21.15.156:6813/30035","state":["exists","up"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":30037},{"type":"v1","addr":"172.21.15.156:6801","nonce":30037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":30037},{"type":"v1","addr":"172.21.15.156:6803","nonce":30037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":30037},{"type":"v1","addr":"172.21.15.156:6807","nonce":30037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":30037},{"type":"v1","addr":"172.21.15.156:6805","nonce":30037}]},"public_addr":"172.21.15.156:6801/30037","cluster_addr":"172.21.15.156:6803/30037","heartbeat_back_addr":"172.21.15.156:6807/30037","heartbeat_front_addr":"172.21.15.156:6805/30037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.781977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.943955+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.806513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.624301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:29:48.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 10 mgrc shutdown 2023-09-29T23:29:48.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 -- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fadc808e830 msgr2=0x7fadc8090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fadc808e830 0x7fadc8090ce0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fadd4009e60 tx=0x7fadd4019000 comp rx=0 tx=0).stop 2023-09-29T23:29:48.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 -- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faddc147c30 msgr2=0x7faddc0a8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faddc147c30 0x7faddc0a8bb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00c4e0 tx=0x7fadcc007e60 comp rx=0 tx=0).stop 2023-09-29T23:29:48.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 -- 172.21.15.143:0/4138802144 shutdown_connections 2023-09-29T23:29:48.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fadc808e830 0x7fadc8090ce0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faddc147c30 0x7faddc0a8bb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faddc0b30b0 0x7faddc1366d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 --2- 172.21.15.143:0/4138802144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faddc06acd0 0x7faddc138bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 -- 172.21.15.143:0/4138802144 >> 172.21.15.143:0/4138802144 conn(0x7faddc067dc0 msgr2=0x7faddc0675c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 -- 172.21.15.143:0/4138802144 shutdown_connections 2023-09-29T23:29:48.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.192+0000 7fade2c78700 1 -- 172.21.15.143:0/4138802144 wait complete. 2023-09-29T23:29:48.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-29T23:29:48.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-29T23:29:48.212 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-29T23:29:48.213 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-29T23:29:48.306 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c8ed7700 1 Processor -- start 2023-09-29T23:29:48.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649e975700 1 Processor -- start 2023-09-29T23:29:48.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c8ed7700 1 -- start start 2023-09-29T23:29:48.307 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649e975700 1 -- start start 2023-09-29T23:29:48.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c8ed7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f73c409f820 0x7f73c41446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.308 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c8ed7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73c4068070 0x7f73c4146c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649e975700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f64980a2140 0x7f6498149b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649e975700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64980a05d0 con 0x7f64980a1d30 2023-09-29T23:29:48.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649e975700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64980a1550 con 0x7f64980a5e20 2023-09-29T23:29:48.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649e975700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64980a61f0 con 0x7f64980a2140 2023-09-29T23:29:48.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c8ed7700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c40a4d50 con 0x7f73c4067ca0 2023-09-29T23:29:48.309 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c8ed7700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c409c320 con 0x7f73c4068070 2023-09-29T23:29:48.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c8ed7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c409c5c0 con 0x7f73c409f820 2023-09-29T23:29:48.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649d172700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f64980a1d30 0x7f6498146420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60674/0 (socket says 172.21.15.143:60674) 2023-09-29T23:29:48.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649d172700 1 -- 172.21.15.143:0/83868562 learned_addr learned my addr 172.21.15.143:0/83868562 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:48.310 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649e174700 1 --2- 172.21.15.143:0/83868562 >> v2:172.21.15.143:3300/0 conn(0x7f64980a2140 0x7f6498149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c2d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f73c4068070 0x7f73c4146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c259c700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f73c4067ca0 0x7f73c409e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.143:47816/0 (socket says 172.21.15.143:47816) 2023-09-29T23:29:48.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c259c700 1 -- 172.21.15.143:0/4173878938 learned_addr learned my addr 172.21.15.143:0/4173878938 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:48.311 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c1d9b700 1 --2- 172.21.15.143:0/4173878938 >> v2:172.21.15.156:3300/0 conn(0x7f73c409f820 0x7f73c41446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649e174700 1 -- 172.21.15.143:0/83868562 >> v1:172.21.15.156:6789/0 conn(0x7f64980a5e20 legacy=0x7f649813e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649e174700 1 -- 172.21.15.143:0/83868562 >> v1:172.21.15.143:6789/0 conn(0x7f64980a1d30 legacy=0x7f6498146420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649e174700 1 -- 172.21.15.143:0/83868562 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649809f910 con 0x7f64980a2140 2023-09-29T23:29:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649c971700 1 -- 172.21.15.143:0/83868562 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 648659853 0 0) 0x7f64980a05d0 con 0x7f64980a1d30 2023-09-29T23:29:48.312 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c2d9d700 1 -- 172.21.15.143:0/4173878938 >> v2:172.21.15.156:3300/0 conn(0x7f73c409f820 msgr2=0x7f73c41446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c2d9d700 1 --2- 172.21.15.143:0/4173878938 >> v2:172.21.15.156:3300/0 conn(0x7f73c409f820 0x7f73c41446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c2d9d700 1 -- 172.21.15.143:0/4173878938 >> v1:172.21.15.156:6789/0 conn(0x7f73c4067ca0 legacy=0x7f73c409e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c2d9d700 1 -- 172.21.15.143:0/4173878938 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c409eb40 con 0x7f73c4068070 2023-09-29T23:29:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649e174700 1 --2- 172.21.15.143:0/83868562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64980a2140 0x7f6498149b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6490009e60 tx=0x7f6490009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fdd254a60210f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c2d9d700 1 --2- 172.21.15.143:0/4173878938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c4068070 0x7f73c4146c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f73c406b130 tx=0x7f73bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6321b0ace76d24 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649c971700 1 -- 172.21.15.143:0/83868562 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f649000e070 con 0x7f64980a2140 2023-09-29T23:29:48.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c159a700 1 -- 172.21.15.143:0/4173878938 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00e070 con 0x7f73c4068070 2023-09-29T23:29:48.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f73c159a700 1 -- 172.21.15.143:0/4173878938 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73bc00bb60 con 0x7f73c4068070 2023-09-29T23:29:48.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649c971700 1 -- 172.21.15.143:0/83868562 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f649000bc30 con 0x7f64980a2140 2023-09-29T23:29:48.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.304+0000 7f649c971700 1 -- 172.21.15.143:0/83868562 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6490012430 con 0x7f64980a2140 2023-09-29T23:29:48.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 -- 172.21.15.143:0/83868562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64980a2140 msgr2=0x7f6498149b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 --2- 172.21.15.143:0/83868562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64980a2140 0x7f6498149b50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6490009e60 tx=0x7f6490009780 comp rx=0 tx=0).stop 2023-09-29T23:29:48.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c159a700 1 -- 172.21.15.143:0/4173878938 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73bc012430 con 0x7f73c4068070 2023-09-29T23:29:48.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 -- 172.21.15.143:0/83868562 shutdown_connections 2023-09-29T23:29:48.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 --2- 172.21.15.143:0/83868562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64980a2140 0x7f6498149b50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 -- 172.21.15.143:0/83868562 >> 172.21.15.143:0/83868562 conn(0x7f6498067da0 msgr2=0x7f64980681a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 -- 172.21.15.143:0/83868562 shutdown_connections 2023-09-29T23:29:48.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 -- 172.21.15.143:0/83868562 wait complete. 2023-09-29T23:29:48.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 -- 172.21.15.143:0/4173878938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c4068070 msgr2=0x7f73c4146c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 --2- 172.21.15.143:0/4173878938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c4068070 0x7f73c4146c20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f73c406b130 tx=0x7f73bc009780 comp rx=0 tx=0).stop 2023-09-29T23:29:48.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 -- 172.21.15.143:0/4173878938 shutdown_connections 2023-09-29T23:29:48.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 --2- 172.21.15.143:0/4173878938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c4068070 0x7f73c4146c20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 --2- 172.21.15.143:0/4173878938 >> v2:172.21.15.156:3300/0 conn(0x7f73c409f820 0x7f73c41446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 -- 172.21.15.143:0/4173878938 >> 172.21.15.143:0/4173878938 conn(0x7f73c40b2ee0 msgr2=0x7f73c40b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 -- 172.21.15.143:0/4173878938 shutdown_connections 2023-09-29T23:29:48.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 -- 172.21.15.143:0/4173878938 wait complete. 2023-09-29T23:29:48.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 Processor -- start 2023-09-29T23:29:48.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 -- start start 2023-09-29T23:29:48.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 Processor -- start 2023-09-29T23:29:48.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 -- start start 2023-09-29T23:29:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f64980a1d30 0x7f6498099850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64980a2140 0x7f649813fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64980a5e20 0x7f6498096c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6498098730 con 0x7f64980a5e20 2023-09-29T23:29:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6498097890 con 0x7f64980a2140 2023-09-29T23:29:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e975700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6498097b50 con 0x7f64980a1d30 2023-09-29T23:29:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73c4067ca0 0x7f73c409a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73c4068070 0x7f73c4096a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c409f820 0x7f73c4096f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c4098090 con 0x7f73c409f820 2023-09-29T23:29:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c409b180 con 0x7f73c4067ca0 2023-09-29T23:29:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c8ed7700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c4142d50 con 0x7f73c4068070 2023-09-29T23:29:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c2d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c409f820 0x7f73c4096f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c1d9b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73c4068070 0x7f73c4096a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c2d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c409f820 0x7f73c4096f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34534/0 (socket says 172.21.15.143:34534) 2023-09-29T23:29:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c2d9d700 1 -- 172.21.15.143:0/3566686800 learned_addr learned my addr 172.21.15.143:0/3566686800 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649d973700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f64980a1d30 0x7f6498099850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e174700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64980a5e20 0x7f6498096c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649d973700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f64980a1d30 0x7f6498099850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60378/0 (socket says 172.21.15.143:60378) 2023-09-29T23:29:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649e174700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64980a5e20 0x7f6498096c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34524/0 (socket says 172.21.15.143:34524) 2023-09-29T23:29:48.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649d973700 1 -- 172.21.15.143:0/2793518631 learned_addr learned my addr 172.21.15.143:0/2793518631 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:48.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c259c700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73c4067ca0 0x7f73c409a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649d172700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64980a2140 0x7f649813fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649d973700 1 -- 172.21.15.143:0/2793518631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64980a2140 msgr2=0x7f649813fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649d973700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f64980a2140 0x7f649813fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649d973700 1 -- 172.21.15.143:0/2793518631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64980a5e20 msgr2=0x7f6498096c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649d973700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f64980a5e20 0x7f6498096c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649d973700 1 -- 172.21.15.143:0/2793518631 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6490009a20 con 0x7f64980a1d30 2023-09-29T23:29:48.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c1d9b700 1 -- 172.21.15.143:0/3566686800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73c4067ca0 msgr2=0x7f73c409a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c1d9b700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f73c4067ca0 0x7f73c409a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c1d9b700 1 -- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c409f820 msgr2=0x7f73c4096f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c1d9b700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c409f820 0x7f73c4096f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c1d9b700 1 -- 172.21.15.143:0/3566686800 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc009a20 con 0x7f73c4068070 2023-09-29T23:29:48.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c2d9d700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73c409f820 0x7f73c4096f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:48.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f649d973700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f64980a1d30 0x7f6498099850 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f649400c590 tx=0x7f6494007f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.308+0000 7f73c1d9b700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73c4068070 0x7f73c4096a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f73b800ea50 tx=0x7f73b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f648dffb700 1 -- 172.21.15.143:0/2793518631 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494011590 con 0x7f64980a1d30 2023-09-29T23:29:48.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f648dffb700 1 -- 172.21.15.143:0/2793518631 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6494011710 con 0x7f64980a1d30 2023-09-29T23:29:48.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f73b27fc700 1 -- 172.21.15.143:0/3566686800 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b800ee90 con 0x7f73c4068070 2023-09-29T23:29:48.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f73b27fc700 1 -- 172.21.15.143:0/3566686800 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b800c880 con 0x7f73c4068070 2023-09-29T23:29:48.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f73c8ed7700 1 -- 172.21.15.143:0/3566686800 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c4097590 con 0x7f73c4068070 2023-09-29T23:29:48.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f73b27fc700 1 -- 172.21.15.143:0/3566686800 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b800cc20 con 0x7f73c4068070 2023-09-29T23:29:48.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f73c8ed7700 1 -- 172.21.15.143:0/3566686800 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c40982c0 con 0x7f73c4068070 2023-09-29T23:29:48.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f648dffb700 1 -- 172.21.15.143:0/2793518631 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6494011ad0 con 0x7f64980a1d30 2023-09-29T23:29:48.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f649e975700 1 -- 172.21.15.143:0/2793518631 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6498140820 con 0x7f64980a1d30 2023-09-29T23:29:48.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f649e975700 1 -- 172.21.15.143:0/2793518631 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649813dce0 con 0x7f64980a1d30 2023-09-29T23:29:48.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f73c8ed7700 1 -- 172.21.15.143:0/3566686800 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f73c405ecb0 con 0x7f73c4068070 2023-09-29T23:29:48.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f73b27fc700 1 -- 172.21.15.143:0/3566686800 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f73b8026480 con 0x7f73c4068070 2023-09-29T23:29:48.330 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f73b27fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:48.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f648dffb700 1 -- 172.21.15.143:0/2793518631 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f649401c460 con 0x7f64980a1d30 2023-09-29T23:29:48.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f648dffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:48.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f649e975700 1 -- 172.21.15.143:0/2793518631 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f649805ecb0 con 0x7f64980a1d30 2023-09-29T23:29:48.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f648dffb700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:48.331 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f648dffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f648dffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f648dffb700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f647c08e830 0x7f647c090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.312+0000 7f649d172700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f647c08e830 0x7f647c090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f648dffb700 1 -- 172.21.15.143:0/2793518631 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f64940b4440 con 0x7f64980a1d30 2023-09-29T23:29:48.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f648dffb700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.156:6808/30035,v1:172.21.15.156:6809/30035] conn(0x7f647c096220 0x7f647c098650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f648dffb700 1 -- 172.21.15.143:0/2793518631 --> [v2:172.21.15.156:6808/30035,v1:172.21.15.156:6809/30035] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f647c098d20 con 0x7f647c096220 2023-09-29T23:29:48.332 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f648dffb700 1 -- 172.21.15.143:0/2793518631 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64940b6a80 con 0x7f64980a1d30 2023-09-29T23:29:48.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73b27fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:48.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73b27fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73b27fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73b27fc700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f73a408e8b0 0x7f73a4090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.333 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73c259c700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f73a408e8b0 0x7f73a4090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f649d172700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f647c08e830 0x7f647c090ce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f648800c8f0 tx=0x7f6488009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73b27fc700 1 -- 172.21.15.143:0/3566686800 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f73b80b51d0 con 0x7f73c4068070 2023-09-29T23:29:48.334 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f649e174700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.156:6808/30035,v1:172.21.15.156:6809/30035] conn(0x7f647c096220 0x7f647c098650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73b27fc700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:6800/27741,v1:172.21.15.143:6801/27741] conn(0x7f73a4096410 0x7f73a4098840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73c259c700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f73a408e8b0 0x7f73a4090d60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f73b4001950 tx=0x7f73b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73b27fc700 1 -- 172.21.15.143:0/3566686800 --> [v2:172.21.15.143:6800/27741,v1:172.21.15.143:6801/27741] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f73a4098f10 con 0x7f73a4096410 2023-09-29T23:29:48.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73b27fc700 1 -- 172.21.15.143:0/3566686800 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73b80b55a0 con 0x7f73c4068070 2023-09-29T23:29:48.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f027c911700 1 Processor -- start 2023-09-29T23:29:48.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73c2d9d700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:6800/27741,v1:172.21.15.143:6801/27741] conn(0x7f73a4096410 0x7f73a4098840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.336 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f027c911700 1 -- start start 2023-09-29T23:29:48.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73c2d9d700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:6800/27741,v1:172.21.15.143:6801/27741] conn(0x7f73a4096410 0x7f73a4098840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f649e174700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.156:6808/30035,v1:172.21.15.156:6809/30035] conn(0x7f647c096220 0x7f647c098650 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f027c911700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02780a5040 0x7f027806abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f027c911700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f027806e840 0x7f0278140800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f027c911700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02780a1960 con 0x7f027806c6a0 2023-09-29T23:29:48.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f027c911700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027806cab0 con 0x7f027806e840 2023-09-29T23:29:48.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f027c911700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027806b9d0 con 0x7f02780a5040 2023-09-29T23:29:48.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02780a5040 0x7f027806abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.338 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02780a5040 0x7f027806abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60396/0 (socket says 172.21.15.143:60396) 2023-09-29T23:29:48.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f0276d9d700 1 -- 172.21.15.143:0/3419353223 learned_addr learned my addr 172.21.15.143:0/3419353223 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:48.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f73b27fc700 1 -- 172.21.15.143:0/3566686800 <== osd.0 v2:172.21.15.143:6800/27741 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f73a4098f10 con 0x7f73a4096410 2023-09-29T23:29:48.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f027759e700 1 --2- 172.21.15.143:0/3419353223 >> v2:172.21.15.156:3300/0 conn(0x7f027806e840 0x7f0278140800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f0276d9d700 1 -- 172.21.15.143:0/3419353223 >> v2:172.21.15.156:3300/0 conn(0x7f027806e840 msgr2=0x7f0278140800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:48.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f0276d9d700 1 --2- 172.21.15.143:0/3419353223 >> v2:172.21.15.156:3300/0 conn(0x7f027806e840 0x7f0278140800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f0276d9d700 1 -- 172.21.15.143:0/3419353223 >> v1:172.21.15.156:6789/0 conn(0x7f027806c6a0 legacy=0x7f02781400f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f0276d9d700 1 -- 172.21.15.143:0/3419353223 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780a5410 con 0x7f02780a5040 2023-09-29T23:29:48.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f0276d9d700 1 --2- 172.21.15.143:0/3419353223 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02780a5040 0x7f027806abc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0270009f20 tx=0x7f0270009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b9036edc409d60 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.316+0000 7f648dffb700 1 -- 172.21.15.143:0/2793518631 <== osd.2 v2:172.21.15.156:6808/30035 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f647c098d20 con 0x7f647c096220 2023-09-29T23:29:48.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f0275d9b700 1 -- 172.21.15.143:0/3419353223 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000e070 con 0x7f02780a5040 2023-09-29T23:29:48.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f0275d9b700 1 -- 172.21.15.143:0/3419353223 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f027000ba80 con 0x7f02780a5040 2023-09-29T23:29:48.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f0275d9b700 1 -- 172.21.15.143:0/3419353223 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02700092e0 con 0x7f02780a5040 2023-09-29T23:29:48.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f027c911700 1 -- 172.21.15.143:0/3419353223 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02780a5040 msgr2=0x7f027806abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f027c911700 1 --2- 172.21.15.143:0/3419353223 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02780a5040 0x7f027806abc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0270009f20 tx=0x7f0270009840 comp rx=0 tx=0).stop 2023-09-29T23:29:48.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f027c911700 1 -- 172.21.15.143:0/3419353223 shutdown_connections 2023-09-29T23:29:48.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f027c911700 1 --2- 172.21.15.143:0/3419353223 >> v2:172.21.15.156:3300/0 conn(0x7f027806e840 0x7f0278140800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f027c911700 1 --2- 172.21.15.143:0/3419353223 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f02780a5040 0x7f027806abc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f027c911700 1 -- 172.21.15.143:0/3419353223 >> 172.21.15.143:0/3419353223 conn(0x7f02780b24b0 msgr2=0x7f02780b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f027c911700 1 -- 172.21.15.143:0/3419353223 shutdown_connections 2023-09-29T23:29:48.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f027c911700 1 -- 172.21.15.143:0/3419353223 wait complete. 2023-09-29T23:29:48.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f027c911700 1 Processor -- start 2023-09-29T23:29:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.320+0000 7f027c911700 1 -- start start 2023-09-29T23:29:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027c911700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f027806c6a0 0x7f027813b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027c911700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f027806e840 0x7f027814ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027c911700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02780a5040 0x7f027814cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027c911700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027809dfe0 con 0x7f027806e840 2023-09-29T23:29:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027c911700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027809a680 con 0x7f02780a5040 2023-09-29T23:29:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027c911700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f027814d600 con 0x7f027806c6a0 2023-09-29T23:29:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f027806e840 0x7f027814ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f027806c6a0 0x7f027813b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f0276d9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f027806c6a0 0x7f027813b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60404/0 (socket says 172.21.15.143:60404) 2023-09-29T23:29:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f027806e840 0x7f027814ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34540/0 (socket says 172.21.15.143:34540) 2023-09-29T23:29:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f0276d9d700 1 -- 172.21.15.143:0/541873811 learned_addr learned my addr 172.21.15.143:0/541873811 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027759e700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02780a5040 0x7f027814cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f0276d9d700 1 -- 172.21.15.143:0/541873811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02780a5040 msgr2=0x7f027814cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f0276d9d700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02780a5040 0x7f027814cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f0276d9d700 1 -- 172.21.15.143:0/541873811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f027806e840 msgr2=0x7f027814ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f0276d9d700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f027806e840 0x7f027814ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f0276d9d700 1 -- 172.21.15.143:0/541873811 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0270009ae0 con 0x7f027806c6a0 2023-09-29T23:29:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027659c700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f027806e840 0x7f027814ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f0276d9d700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f027806c6a0 0x7f027813b5e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f027000b6c0 tx=0x7f0270018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f026effd700 1 -- 172.21.15.143:0/541873811 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02700078f0 con 0x7f027806c6a0 2023-09-29T23:29:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f026effd700 1 -- 172.21.15.143:0/541873811 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0270007a50 con 0x7f027806c6a0 2023-09-29T23:29:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027c911700 1 -- 172.21.15.143:0/541873811 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f027814d8c0 con 0x7f027806c6a0 2023-09-29T23:29:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f026effd700 1 -- 172.21.15.143:0/541873811 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0270007e10 con 0x7f027806c6a0 2023-09-29T23:29:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027c911700 1 -- 172.21.15.143:0/541873811 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027809da90 con 0x7f027806c6a0 2023-09-29T23:29:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7fd180a4d700 1 Processor -- start 2023-09-29T23:29:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7fd180a4d700 1 -- start start 2023-09-29T23:29:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.324+0000 7f027c911700 1 -- 172.21.15.143:0/541873811 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f027805ecb0 con 0x7f027806c6a0 2023-09-29T23:29:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd180a4d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd17c0a5040 0x7fd17c0a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd180a4d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c0a02c0 con 0x7fd17c06e020 2023-09-29T23:29:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd180a4d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c0a5410 con 0x7fd17c13d0b0 2023-09-29T23:29:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd180a4d700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17c09f4f0 con 0x7fd17c0a5040 2023-09-29T23:29:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd17affd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd17c06e020 0x7fd17c06e450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60682/0 (socket says 172.21.15.143:60682) 2023-09-29T23:29:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd17affd700 1 -- 172.21.15.143:0/2162062599 learned_addr learned my addr 172.21.15.143:0/2162062599 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd17b7fe700 1 --2- 172.21.15.143:0/2162062599 >> v2:172.21.15.143:3300/0 conn(0x7fd17c0a5040 0x7fd17c0a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd17b7fe700 1 -- 172.21.15.143:0/2162062599 >> v1:172.21.15.156:6789/0 conn(0x7fd17c13d0b0 legacy=0x7fd17c13d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd17b7fe700 1 -- 172.21.15.143:0/2162062599 >> v1:172.21.15.143:6789/0 conn(0x7fd17c06e020 legacy=0x7fd17c06e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd17b7fe700 1 -- 172.21.15.143:0/2162062599 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c09f7b0 con 0x7fd17c0a5040 2023-09-29T23:29:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f026effd700 1 -- 172.21.15.143:0/541873811 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f0270003040 con 0x7f027806c6a0 2023-09-29T23:29:48.351 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f026effd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd17b7fe700 1 --2- 172.21.15.143:0/2162062599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd17c0a5040 0x7fd17c0a11e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd1700098e0 tx=0x7fd1700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5c22f12e8629a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.352 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd17a7fc700 1 -- 172.21.15.143:0/2162062599 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd170009e80 con 0x7fd17c0a5040 2023-09-29T23:29:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd17a7fc700 1 -- 172.21.15.143:0/2162062599 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17000b760 con 0x7fd17c0a5040 2023-09-29T23:29:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd17a7fc700 1 -- 172.21.15.143:0/2162062599 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17000bae0 con 0x7fd17c0a5040 2023-09-29T23:29:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f026effd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f026effd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f026effd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f026effd700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f025808ea00 0x7f0258090eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f027659c700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f025808ea00 0x7f0258090eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f026effd700 1 -- 172.21.15.143:0/541873811 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f02700b5690 con 0x7f027806c6a0 2023-09-29T23:29:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f026effd700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.156:6816/30036,v1:172.21.15.156:6817/30036] conn(0x7f0258096560 0x7f0258098990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f026effd700 1 -- 172.21.15.143:0/541873811 --> [v2:172.21.15.156:6816/30036,v1:172.21.15.156:6817/30036] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0258099060 con 0x7f0258096560 2023-09-29T23:29:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f026effd700 1 -- 172.21.15.143:0/541873811 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02700b7b70 con 0x7f027806c6a0 2023-09-29T23:29:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f027659c700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f025808ea00 0x7f0258090eb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0278099b50 tx=0x7f0268000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f027759e700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.156:6816/30036,v1:172.21.15.156:6817/30036] conn(0x7f0258096560 0x7f0258098990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd180a4d700 1 -- 172.21.15.143:0/2162062599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd17c0a5040 msgr2=0x7fd17c0a11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd180a4d700 1 --2- 172.21.15.143:0/2162062599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd17c0a5040 0x7fd17c0a11e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd1700098e0 tx=0x7fd1700091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd180a4d700 1 -- 172.21.15.143:0/2162062599 shutdown_connections 2023-09-29T23:29:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd180a4d700 1 --2- 172.21.15.143:0/2162062599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd17c0a5040 0x7fd17c0a11e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd180a4d700 1 -- 172.21.15.143:0/2162062599 >> 172.21.15.143:0/2162062599 conn(0x7fd17c0b24b0 msgr2=0x7fd17c0b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd180a4d700 1 -- 172.21.15.143:0/2162062599 shutdown_connections 2023-09-29T23:29:48.356 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7fd180a4d700 1 -- 172.21.15.143:0/2162062599 wait complete. 2023-09-29T23:29:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.328+0000 7f027759e700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.156:6816/30036,v1:172.21.15.156:6817/30036] conn(0x7f0258096560 0x7f0258098990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7f026effd700 1 -- 172.21.15.143:0/541873811 <== osd.1 v2:172.21.15.156:6816/30036 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0258099060 con 0x7f0258096560 2023-09-29T23:29:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd180a4d700 1 Processor -- start 2023-09-29T23:29:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7f73c8ed7700 1 -- 172.21.15.143:0/3566686800 --> [v2:172.21.15.143:6800/27741,v1:172.21.15.143:6801/27741] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f73c404b5c0 con 0x7f73a4096410 2023-09-29T23:29:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd180a4d700 1 -- start start 2023-09-29T23:29:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd180a4d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd17c06e020 0x7fd17c0a5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd180a4d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c0a5040 0x7fd17c142720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd180a4d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd17c13d0b0 0x7fd17c142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd180a4d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c143360 con 0x7fd17c06e020 2023-09-29T23:29:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd180a4d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c146130 con 0x7fd17c0a5040 2023-09-29T23:29:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd180a4d700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c09eb60 con 0x7fd17c13d0b0 2023-09-29T23:29:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd17b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd17c06e020 0x7fd17c0a5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd17bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd17c13d0b0 0x7fd17c142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd17b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd17c06e020 0x7fd17c0a5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34562/0 (socket says 172.21.15.143:34562) 2023-09-29T23:29:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd17b7fe700 1 -- 172.21.15.143:0/2602783004 learned_addr learned my addr 172.21.15.143:0/2602783004 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd17affd700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c0a5040 0x7fd17c142720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd17bfff700 1 -- 172.21.15.143:0/2602783004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c0a5040 msgr2=0x7fd17c142720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd17bfff700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd17c0a5040 0x7fd17c142720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd17bfff700 1 -- 172.21.15.143:0/2602783004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd17c06e020 msgr2=0x7fd17c0a5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd17bfff700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd17c06e020 0x7fd17c0a5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd17bfff700 1 -- 172.21.15.143:0/2602783004 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd170009420 con 0x7fd17c13d0b0 2023-09-29T23:29:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd17bfff700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd17c13d0b0 0x7fd17c142c60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00c3c0 tx=0x7fd16c007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd1637fe700 1 -- 172.21.15.143:0/2602783004 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c012420 con 0x7fd17c13d0b0 2023-09-29T23:29:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd1637fe700 1 -- 172.21.15.143:0/2602783004 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c0125a0 con 0x7fd17c13d0b0 2023-09-29T23:29:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd180a4d700 1 -- 172.21.15.143:0/2602783004 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c09ed10 con 0x7fd17c13d0b0 2023-09-29T23:29:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd1637fe700 1 -- 172.21.15.143:0/2602783004 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c012960 con 0x7fd17c13d0b0 2023-09-29T23:29:48.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7fd180a4d700 1 -- 172.21.15.143:0/2602783004 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c0b1f80 con 0x7fd17c13d0b0 2023-09-29T23:29:48.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.332+0000 7f73b27fc700 1 -- 172.21.15.143:0/3566686800 <== osd.0 v2:172.21.15.143:6800/27741 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f73c404b5c0 con 0x7f73a4096410 2023-09-29T23:29:48.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c8ed7700 1 -- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:6800/27741,v1:172.21.15.143:6801/27741] conn(0x7f73a4096410 msgr2=0x7f73a4098840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.361 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c8ed7700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:6800/27741,v1:172.21.15.143:6801/27741] conn(0x7f73a4096410 0x7f73a4098840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c8ed7700 10 mgrc shutdown 2023-09-29T23:29:48.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c8ed7700 1 -- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f73a408e8b0 msgr2=0x7f73a4090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c8ed7700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f73a408e8b0 0x7f73a4090d60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f73b4001950 tx=0x7f73b4009040 comp rx=0 tx=0).stop 2023-09-29T23:29:48.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c8ed7700 1 -- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73c4068070 msgr2=0x7f73c4096a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c8ed7700 1 --2- 172.21.15.143:0/3566686800 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f73c4068070 0x7f73c4096a10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f73b800ea50 tx=0x7f73b80091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c2d9d700 1 -- 172.21.15.143:0/3566686800 reap_dead start 2023-09-29T23:29:48.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c8ed7700 1 -- 172.21.15.143:0/3566686800 shutdown_connections 2023-09-29T23:29:48.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c8ed7700 1 -- 172.21.15.143:0/3566686800 >> 172.21.15.143:0/3566686800 conn(0x7f73c40b2ee0 msgr2=0x7f73c40a5110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7fd180a4d700 1 -- 172.21.15.143:0/2602783004 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd17c05ecb0 con 0x7fd17c13d0b0 2023-09-29T23:29:48.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c8ed7700 1 -- 172.21.15.143:0/3566686800 shutdown_connections 2023-09-29T23:29:48.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7f73c8ed7700 1 -- 172.21.15.143:0/3566686800 wait complete. 2023-09-29T23:29:48.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7fd1637fe700 1 -- 172.21.15.143:0/2602783004 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fd16c01b3b0 con 0x7fd17c13d0b0 2023-09-29T23:29:48.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7fd1637fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:48.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7fd1637fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:48.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7fd1637fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7fd1637fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7fd1637fe700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd16808e8b0 0x7fd168090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7fd1637fe700 1 -- 172.21.15.143:0/2602783004 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7fd16c0b3cd0 con 0x7fd17c13d0b0 2023-09-29T23:29:48.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7fd1637fe700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.156:6800/30037,v1:172.21.15.156:6801/30037] conn(0x7fd168096410 0x7fd168098840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7fd1637fe700 1 -- 172.21.15.143:0/2602783004 --> [v2:172.21.15.156:6800/30037,v1:172.21.15.156:6801/30037] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd168098f10 con 0x7fd168096410 2023-09-29T23:29:48.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.336+0000 7fd1637fe700 1 -- 172.21.15.143:0/2602783004 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd16c0b6310 con 0x7fd17c13d0b0 2023-09-29T23:29:48.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7fd17affd700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.156:6800/30037,v1:172.21.15.156:6801/30037] conn(0x7fd168096410 0x7fd168098840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 1 -- 172.21.15.143:0/2793518631 --> [v2:172.21.15.156:6808/30035,v1:172.21.15.156:6809/30035] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6498097380 con 0x7f647c096220 2023-09-29T23:29:48.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7fd17affd700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.156:6800/30037,v1:172.21.15.156:6801/30037] conn(0x7fd168096410 0x7fd168098840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f648dffb700 1 -- 172.21.15.143:0/2793518631 <== osd.2 v2:172.21.15.156:6808/30035 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6498097380 con 0x7f647c096220 2023-09-29T23:29:48.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 1 -- 172.21.15.143:0/2793518631 >> [v2:172.21.15.156:6808/30035,v1:172.21.15.156:6809/30035] conn(0x7f647c096220 msgr2=0x7f647c098650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.156:6808/30035,v1:172.21.15.156:6809/30035] conn(0x7f647c096220 0x7f647c098650 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 10 mgrc shutdown 2023-09-29T23:29:48.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 1 -- 172.21.15.143:0/2793518631 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f647c08e830 msgr2=0x7f647c090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f647c08e830 0x7f647c090ce0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f648800c8f0 tx=0x7f6488009040 comp rx=0 tx=0).stop 2023-09-29T23:29:48.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 1 -- 172.21.15.143:0/2793518631 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f64980a1d30 msgr2=0x7f6498099850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 1 --2- 172.21.15.143:0/2793518631 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f64980a1d30 0x7f6498099850 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f649400c590 tx=0x7f6494007f30 comp rx=0 tx=0).stop 2023-09-29T23:29:48.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e174700 1 -- 172.21.15.143:0/2793518631 reap_dead start 2023-09-29T23:29:48.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 1 -- 172.21.15.143:0/2793518631 shutdown_connections 2023-09-29T23:29:48.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 1 -- 172.21.15.143:0/2793518631 >> 172.21.15.143:0/2793518631 conn(0x7f6498067da0 msgr2=0x7f64980673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 1 -- 172.21.15.143:0/2793518631 shutdown_connections 2023-09-29T23:29:48.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7f649e975700 1 -- 172.21.15.143:0/2793518631 wait complete. 2023-09-29T23:29:48.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.340+0000 7fd1637fe700 1 -- 172.21.15.143:0/2602783004 <== osd.3 v2:172.21.15.156:6800/30037 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd168098f10 con 0x7fd168096410 2023-09-29T23:29:48.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7fd17b7fe700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd16808e8b0 0x7fd168090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7fd17b7fe700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd16808e8b0 0x7fd168090d60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd1700093a0 tx=0x7fd170016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7f027c911700 1 -- 172.21.15.143:0/541873811 --> [v2:172.21.15.156:6816/30036,v1:172.21.15.156:6817/30036] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f027813e2b0 con 0x7f0258096560 2023-09-29T23:29:48.370 INFO:teuthology.orchestra.run.smithi143.stdout:38654705672 2023-09-29T23:29:48.370 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-29T23:29:48.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7f026effd700 1 -- 172.21.15.143:0/541873811 <== osd.1 v2:172.21.15.156:6816/30036 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f027813e2b0 con 0x7f0258096560 2023-09-29T23:29:48.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7f027c911700 1 -- 172.21.15.143:0/541873811 >> [v2:172.21.15.156:6816/30036,v1:172.21.15.156:6817/30036] conn(0x7f0258096560 msgr2=0x7f0258098990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7f027c911700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.156:6816/30036,v1:172.21.15.156:6817/30036] conn(0x7f0258096560 0x7f0258098990 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7f027c911700 10 mgrc shutdown 2023-09-29T23:29:48.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7f027c911700 1 -- 172.21.15.143:0/541873811 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f025808ea00 msgr2=0x7f0258090eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7f027c911700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f025808ea00 0x7f0258090eb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0278099b50 tx=0x7f0268000bc0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7f027c911700 1 -- 172.21.15.143:0/541873811 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f027806c6a0 msgr2=0x7f027813b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7f027c911700 1 --2- 172.21.15.143:0/541873811 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f027806c6a0 0x7f027813b5e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f027000b6c0 tx=0x7f0270018000 comp rx=0 tx=0).stop 2023-09-29T23:29:48.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.348+0000 7f027759e700 1 -- 172.21.15.143:0/541873811 reap_dead start 2023-09-29T23:29:48.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.352+0000 7f027c911700 1 -- 172.21.15.143:0/541873811 shutdown_connections 2023-09-29T23:29:48.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.352+0000 7f027c911700 1 -- 172.21.15.143:0/541873811 >> 172.21.15.143:0/541873811 conn(0x7f02780b24b0 msgr2=0x7f027813c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.352+0000 7f027c911700 1 -- 172.21.15.143:0/541873811 shutdown_connections 2023-09-29T23:29:48.374 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.352+0000 7f027c911700 1 -- 172.21.15.143:0/541873811 wait complete. 2023-09-29T23:29:48.374 INFO:teuthology.orchestra.run.smithi143.stdout:38654705671 2023-09-29T23:29:48.375 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-29T23:29:48.375 INFO:teuthology.orchestra.run.smithi143.stdout:38654705671 2023-09-29T23:29:48.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-29T23:29:48.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.360+0000 7fd180a4d700 1 -- 172.21.15.143:0/2602783004 --> [v2:172.21.15.156:6800/30037,v1:172.21.15.156:6801/30037] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd17c0a0510 con 0x7fd168096410 2023-09-29T23:29:48.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.360+0000 7fd1637fe700 1 -- 172.21.15.143:0/2602783004 <== osd.3 v2:172.21.15.156:6800/30037 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd17c0a0510 con 0x7fd168096410 2023-09-29T23:29:48.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd180a4d700 1 -- 172.21.15.143:0/2602783004 >> [v2:172.21.15.156:6800/30037,v1:172.21.15.156:6801/30037] conn(0x7fd168096410 msgr2=0x7fd168098840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd180a4d700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.156:6800/30037,v1:172.21.15.156:6801/30037] conn(0x7fd168096410 0x7fd168098840 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd180a4d700 10 mgrc shutdown 2023-09-29T23:29:48.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd180a4d700 1 -- 172.21.15.143:0/2602783004 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd16808e8b0 msgr2=0x7fd168090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd180a4d700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd16808e8b0 0x7fd168090d60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd1700093a0 tx=0x7fd170016000 comp rx=0 tx=0).stop 2023-09-29T23:29:48.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd180a4d700 1 -- 172.21.15.143:0/2602783004 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd17c13d0b0 msgr2=0x7fd17c142c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd180a4d700 1 --2- 172.21.15.143:0/2602783004 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd17c13d0b0 0x7fd17c142c60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00c3c0 tx=0x7fd16c007ec0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd17bfff700 1 -- 172.21.15.143:0/2602783004 reap_dead start 2023-09-29T23:29:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd180a4d700 1 -- 172.21.15.143:0/2602783004 shutdown_connections 2023-09-29T23:29:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd180a4d700 1 -- 172.21.15.143:0/2602783004 >> 172.21.15.143:0/2602783004 conn(0x7fd17c0b24b0 msgr2=0x7fd17c13b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd180a4d700 1 -- 172.21.15.143:0/2602783004 shutdown_connections 2023-09-29T23:29:48.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.364+0000 7fd180a4d700 1 -- 172.21.15.143:0/2602783004 wait complete. 2023-09-29T23:29:48.380 INFO:teuthology.orchestra.run.smithi143.stdout:38654705672 2023-09-29T23:29:48.380 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-29T23:29:48.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39db3df700 1 Processor -- start 2023-09-29T23:29:48.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39db3df700 1 -- start start 2023-09-29T23:29:48.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39db3df700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f39d40b3810 0x7f39d406ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39db3df700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f39d406da20 0x7f39d413f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39db3df700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d406c5d0 con 0x7f39d40b3be0 2023-09-29T23:29:48.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39db3df700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d406cfa0 con 0x7f39d406da20 2023-09-29T23:29:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39db3df700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d41407d0 con 0x7f39d40b3810 2023-09-29T23:29:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39d897a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f39d406da20 0x7f39d413f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39d997c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f39d40b3be0 0x7f39d413fdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60690/0 (socket says 172.21.15.143:60690) 2023-09-29T23:29:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39d997c700 1 -- 172.21.15.143:0/2424752749 learned_addr learned my addr 172.21.15.143:0/2424752749 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39d917b700 1 --2- 172.21.15.143:0/2424752749 >> v2:172.21.15.156:3300/0 conn(0x7f39d40b3810 0x7f39d406ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39cbfff700 1 -- 172.21.15.143:0/2424752749 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247627078 0 0) 0x7f39d406c5d0 con 0x7f39d40b3be0 2023-09-29T23:29:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39cbfff700 1 -- 172.21.15.143:0/2424752749 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39c0003570 con 0x7f39d40b3be0 2023-09-29T23:29:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39d897a700 1 -- 172.21.15.143:0/2424752749 >> v2:172.21.15.156:3300/0 conn(0x7f39d40b3810 msgr2=0x7f39d406ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39d897a700 1 --2- 172.21.15.143:0/2424752749 >> v2:172.21.15.156:3300/0 conn(0x7f39d40b3810 0x7f39d406ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39d897a700 1 -- 172.21.15.143:0/2424752749 >> v1:172.21.15.143:6789/0 conn(0x7f39d40b3be0 legacy=0x7f39d413fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39d897a700 1 -- 172.21.15.143:0/2424752749 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d4140a90 con 0x7f39d406da20 2023-09-29T23:29:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39d897a700 1 --2- 172.21.15.143:0/2424752749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d406da20 0x7f39d413f870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39d404c030 tx=0x7f39c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facd1b047420f032 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39cbfff700 1 -- 172.21.15.143:0/2424752749 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4011630 con 0x7f39d406da20 2023-09-29T23:29:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39cbfff700 1 -- 172.21.15.143:0/2424752749 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c40117b0 con 0x7f39d406da20 2023-09-29T23:29:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.500+0000 7f39cbfff700 1 -- 172.21.15.143:0/2424752749 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c4011b70 con 0x7f39d406da20 2023-09-29T23:29:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 -- 172.21.15.143:0/2424752749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d406da20 msgr2=0x7f39d413f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 --2- 172.21.15.143:0/2424752749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d406da20 0x7f39d413f870 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f39d404c030 tx=0x7f39c40091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 -- 172.21.15.143:0/2424752749 shutdown_connections 2023-09-29T23:29:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 --2- 172.21.15.143:0/2424752749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d406da20 0x7f39d413f870 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 --2- 172.21.15.143:0/2424752749 >> v2:172.21.15.156:3300/0 conn(0x7f39d40b3810 0x7f39d406ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 -- 172.21.15.143:0/2424752749 >> 172.21.15.143:0/2424752749 conn(0x7f39d4069820 msgr2=0x7f39d4069c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 -- 172.21.15.143:0/2424752749 shutdown_connections 2023-09-29T23:29:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 -- 172.21.15.143:0/2424752749 wait complete. 2023-09-29T23:29:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 Processor -- start 2023-09-29T23:29:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 -- start start 2023-09-29T23:29:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d406da20 0x7f39d411b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39d40b3810 0x7f39d4147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f39d40b3be0 0x7f39d413cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40adaf0 con 0x7f39d406da20 2023-09-29T23:29:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40addb0 con 0x7f39d40b3810 2023-09-29T23:29:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39db3df700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d40ae070 con 0x7f39d40b3be0 2023-09-29T23:29:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39d917b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d406da20 0x7f39d411b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39d917b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d406da20 0x7f39d411b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34584/0 (socket says 172.21.15.143:34584) 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39d997c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f39d40b3be0 0x7f39d413cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39d917b700 1 -- 172.21.15.143:0/1755262302 learned_addr learned my addr 172.21.15.143:0/1755262302 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39d897a700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39d40b3810 0x7f39d4147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39d917b700 1 -- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f39d40b3be0 msgr2=0x7f39d413cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39d917b700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f39d40b3be0 0x7f39d413cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39d917b700 1 -- 172.21.15.143:0/1755262302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39d40b3810 msgr2=0x7f39d4147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39d917b700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39d40b3810 0x7f39d4147330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39d917b700 1 -- 172.21.15.143:0/1755262302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c4009420 con 0x7f39d406da20 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.504+0000 7f39d997c700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f39d40b3be0 0x7f39d413cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39d917b700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d406da20 0x7f39d411b400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f39d000ec80 tx=0x7f39d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39c97fa700 1 -- 172.21.15.143:0/1755262302 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d000c850 con 0x7f39d406da20 2023-09-29T23:29:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39c97fa700 1 -- 172.21.15.143:0/1755262302 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d000c9b0 con 0x7f39d406da20 2023-09-29T23:29:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39db3df700 1 -- 172.21.15.143:0/1755262302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d40aea40 con 0x7f39d406da20 2023-09-29T23:29:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39c97fa700 1 -- 172.21.15.143:0/1755262302 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d000cd50 con 0x7f39d406da20 2023-09-29T23:29:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39db3df700 1 -- 172.21.15.143:0/1755262302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d413d700 con 0x7f39d406da20 2023-09-29T23:29:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39db3df700 1 -- 172.21.15.143:0/1755262302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d405ecb0 con 0x7f39d406da20 2023-09-29T23:29:48.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39c97fa700 1 -- 172.21.15.143:0/1755262302 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f39d0013680 con 0x7f39d406da20 2023-09-29T23:29:48.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39c97fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39c97fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39c97fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39c97fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.508+0000 7f39c97fa700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39b808e8e0 0x7f39b8090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.512+0000 7f39c97fa700 1 -- 172.21.15.143:0/1755262302 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f39d00b5270 con 0x7f39d406da20 2023-09-29T23:29:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.512+0000 7f39d897a700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39b808e8e0 0x7f39b8090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.512+0000 7f39d897a700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39b808e8e0 0x7f39b8090d90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39d404c030 tx=0x7f39c4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 Processor -- start 2023-09-29T23:29:48.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 -- start start 2023-09-29T23:29:48.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2b7c0a5e20 0x7f2b7c09fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c1436e0 0x7f2b7c143b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7c09ec70 con 0x7f2b7c135f80 2023-09-29T23:29:48.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b7c09d130 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c09f0c0 con 0x7f2b7c1436e0 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c1436e0 0x7f2b7c143b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7affd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2b7c135f80 0x7f2b7c136380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60698/0 (socket says 172.21.15.143:60698) 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7affd700 1 -- 172.21.15.143:0/3408814965 learned_addr learned my addr 172.21.15.143:0/3408814965 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7b7fe700 1 --2- 172.21.15.143:0/3408814965 >> v2:172.21.15.156:3300/0 conn(0x7f2b7c0a5e20 0x7f2b7c09fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7bfff700 1 -- 172.21.15.143:0/3408814965 >> v2:172.21.15.156:3300/0 conn(0x7f2b7c0a5e20 msgr2=0x7f2b7c09fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7bfff700 1 --2- 172.21.15.143:0/3408814965 >> v2:172.21.15.156:3300/0 conn(0x7f2b7c0a5e20 0x7f2b7c09fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7bfff700 1 -- 172.21.15.143:0/3408814965 >> v1:172.21.15.143:6789/0 conn(0x7f2b7c135f80 legacy=0x7f2b7c136380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7bfff700 1 -- 172.21.15.143:0/3408814965 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c0a61f0 con 0x7f2b7c1436e0 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7bfff700 1 --2- 172.21.15.143:0/3408814965 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c1436e0 0x7f2b7c143b60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2b700098e0 tx=0x7f2b700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd80aac19e9571a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7a7fc700 1 -- 172.21.15.143:0/3408814965 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70009e80 con 0x7f2b7c1436e0 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7a7fc700 1 -- 172.21.15.143:0/3408814965 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7000b860 con 0x7f2b7c1436e0 2023-09-29T23:29:48.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b7a7fc700 1 -- 172.21.15.143:0/3408814965 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7000bbe0 con 0x7f2b7c1436e0 2023-09-29T23:29:48.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 -- 172.21.15.143:0/3408814965 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c1436e0 msgr2=0x7f2b7c143b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 --2- 172.21.15.143:0/3408814965 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c1436e0 0x7f2b7c143b60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2b700098e0 tx=0x7f2b700091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 -- 172.21.15.143:0/3408814965 shutdown_connections 2023-09-29T23:29:48.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 --2- 172.21.15.143:0/3408814965 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c1436e0 0x7f2b7c143b60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 --2- 172.21.15.143:0/3408814965 >> v2:172.21.15.156:3300/0 conn(0x7f2b7c0a5e20 0x7f2b7c09fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 -- 172.21.15.143:0/3408814965 >> 172.21.15.143:0/3408814965 conn(0x7f2b7c067da0 msgr2=0x7f2b7c0681a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 -- 172.21.15.143:0/3408814965 shutdown_connections 2023-09-29T23:29:48.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.516+0000 7f2b80f1c700 1 -- 172.21.15.143:0/3408814965 wait complete. 2023-09-29T23:29:48.522 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f39c97fa700 1 -- 172.21.15.143:0/1755262302 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f39d0085ca0 con 0x7f39d406da20 2023-09-29T23:29:48.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b80f1c700 1 Processor -- start 2023-09-29T23:29:48.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b80f1c700 1 -- start start 2023-09-29T23:29:48.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b80f1c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c0a5e20 0x7f2b7c09b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b80f1c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c135f80 0x7f2b7c06df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b80f1c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b7c1436e0 0x7f2b7c153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.524 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b80f1c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c06e990 con 0x7f2b7c1436e0 2023-09-29T23:29:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b80f1c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c09ab40 con 0x7f2b7c135f80 2023-09-29T23:29:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b80f1c700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c067720 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b7bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b7c1436e0 0x7f2b7c153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b7b7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c0a5e20 0x7f2b7c09b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b7bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b7c1436e0 0x7f2b7c153250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34600/0 (socket says 172.21.15.143:34600) 2023-09-29T23:29:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b7bfff700 1 -- 172.21.15.143:0/759353236 learned_addr learned my addr 172.21.15.143:0/759353236 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b7b7fe700 1 -- 172.21.15.143:0/759353236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c135f80 msgr2=0x7f2b7c06df20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-29T23:29:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b7b7fe700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c135f80 0x7f2b7c06df20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b7b7fe700 1 -- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b7c1436e0 msgr2=0x7f2b7c153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b7b7fe700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b7c1436e0 0x7f2b7c153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b7b7fe700 1 -- 172.21.15.143:0/759353236 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b70009420 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b7b7fe700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c0a5e20 0x7f2b7c09b570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2b74007f50 tx=0x7f2b740097e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b637fe700 1 -- 172.21.15.143:0/759353236 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400ebc0 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b637fe700 1 -- 172.21.15.143:0/759353236 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7400ed20 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b637fe700 1 -- 172.21.15.143:0/759353236 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b74014650 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b80f1c700 1 -- 172.21.15.143:0/759353236 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c14fca0 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.526 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.520+0000 7f2b80f1c700 1 -- 172.21.15.143:0/759353236 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c06dab0 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f2b80f1c700 1 -- 172.21.15.143:0/759353236 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c05ecb0 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f2b637fe700 1 -- 172.21.15.143:0/759353236 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f2b74014a80 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f2b637fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:48.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f2b637fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:48.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f2b637fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f2b637fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f2b637fe700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2b6808e8b0 0x7f2b68090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f2b637fe700 1 -- 172.21.15.143:0/759353236 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f2b740b4e50 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f2b7affd700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2b6808e8b0 0x7f2b68090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f2b7affd700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2b6808e8b0 0x7f2b68090d60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0058d0 tx=0x7f2b6c007ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f12f9e5f700 1 Processor -- start 2023-09-29T23:29:48.528 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.524+0000 7f12f9e5f700 1 -- start start 2023-09-29T23:29:48.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f9e5f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f12f406f060 0x7f12f409efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f9e5f700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409fb40 0x7f12f4148760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f9e5f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f41413c0 con 0x7f12f409f520 2023-09-29T23:29:48.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f9e5f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f406f430 con 0x7f12f406f060 2023-09-29T23:29:48.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f9e5f700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f41415f0 con 0x7f12f409fb40 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f3fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f12f409f520 0x7f12f413d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60702/0 (socket says 172.21.15.143:60702) 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f8e5d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f12f406f060 0x7f12f409efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f3fff700 1 -- 172.21.15.143:0/82033974 learned_addr learned my addr 172.21.15.143:0/82033974 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f965e700 1 --2- 172.21.15.143:0/82033974 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409fb40 0x7f12f4148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f37fe700 1 -- 172.21.15.143:0/82033974 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549305582 0 0) 0x7f12f41413c0 con 0x7f12f409f520 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f37fe700 1 -- 172.21.15.143:0/82033974 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12dc003570 con 0x7f12f409f520 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f8e5d700 1 -- 172.21.15.143:0/82033974 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409fb40 msgr2=0x7f12f4148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f8e5d700 1 --2- 172.21.15.143:0/82033974 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409fb40 0x7f12f4148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f37fe700 1 -- 172.21.15.143:0/82033974 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1368410380 0 0) 0x7f12dc003570 con 0x7f12f409f520 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f8e5d700 1 -- 172.21.15.143:0/82033974 >> v1:172.21.15.143:6789/0 conn(0x7f12f409f520 legacy=0x7f12f413d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f8e5d700 1 -- 172.21.15.143:0/82033974 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f41418b0 con 0x7f12f406f060 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f965e700 1 --2- 172.21.15.143:0/82033974 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409fb40 0x7f12f4148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:48.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f8e5d700 1 --2- 172.21.15.143:0/82033974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12f406f060 0x7f12f409efe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009e60 tx=0x7f12ec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127c8d96147503f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f37fe700 1 -- 172.21.15.143:0/82033974 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00e070 con 0x7f12f406f060 2023-09-29T23:29:48.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f37fe700 1 -- 172.21.15.143:0/82033974 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec00b830 con 0x7f12f406f060 2023-09-29T23:29:48.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f37fe700 1 -- 172.21.15.143:0/82033974 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e8001470 con 0x7f12f406f060 2023-09-29T23:29:48.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f9e5f700 1 -- 172.21.15.143:0/82033974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12f406f060 msgr2=0x7f12f409efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f12f9e5f700 1 --2- 172.21.15.143:0/82033974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12f406f060 0x7f12f409efe0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009e60 tx=0x7f12ec009780 comp rx=0 tx=0).stop 2023-09-29T23:29:48.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.528+0000 7f2b637fe700 1 -- 172.21.15.143:0/759353236 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2b74085880 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 Processor -- start 2023-09-29T23:29:48.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 -- 172.21.15.143:0/82033974 shutdown_connections 2023-09-29T23:29:48.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 --2- 172.21.15.143:0/82033974 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409fb40 0x7f12f4148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 --2- 172.21.15.143:0/82033974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12f406f060 0x7f12f409efe0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 -- 172.21.15.143:0/82033974 >> 172.21.15.143:0/82033974 conn(0x7f12f40b2c70 msgr2=0x7f12f40b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 -- 172.21.15.143:0/82033974 shutdown_connections 2023-09-29T23:29:48.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 -- 172.21.15.143:0/82033974 wait complete. 2023-09-29T23:29:48.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 -- start start 2023-09-29T23:29:48.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 Processor -- start 2023-09-29T23:29:48.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1d4006ee30 0x7f1d400a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d4006f200 0x7f1d400b0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d400a5330 con 0x7f1d4009c650 2023-09-29T23:29:48.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d400a55f0 con 0x7f1d4006f200 2023-09-29T23:29:48.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d4009cab0 con 0x7f1d4006ee30 2023-09-29T23:29:48.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 -- start start 2023-09-29T23:29:48.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d4006f200 0x7f1d400b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f1d4006f200 0x7f1d400b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34620/0 (socket says 172.21.15.143:34620) 2023-09-29T23:29:48.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3f7fe700 1 -- 172.21.15.143:0/829104847 learned_addr learned my addr 172.21.15.143:0/829104847 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:48.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3ffff700 1 --2- 172.21.15.143:0/829104847 >> v2:172.21.15.156:3300/0 conn(0x7f1d4006ee30 0x7f1d400a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3f7fe700 1 -- 172.21.15.143:0/829104847 >> v2:172.21.15.156:3300/0 conn(0x7f1d4006ee30 msgr2=0x7f1d400a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:48.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3f7fe700 1 --2- 172.21.15.143:0/829104847 >> v2:172.21.15.156:3300/0 conn(0x7f1d4006ee30 0x7f1d400a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3f7fe700 1 -- 172.21.15.143:0/829104847 >> v1:172.21.15.156:6789/0 conn(0x7f1d4009c650 legacy=0x7f1d40067ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:48.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3f7fe700 1 -- 172.21.15.143:0/829104847 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4009c0f0 con 0x7f1d4006f200 2023-09-29T23:29:48.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3f7fe700 1 --2- 172.21.15.143:0/829104847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4006f200 0x7f1d400b0590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1d40068e60 tx=0x7f1d30009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43f22cae911bc5a server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3effd700 1 -- 172.21.15.143:0/829104847 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000b820 con 0x7f1d4006f200 2023-09-29T23:29:48.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3effd700 1 -- 172.21.15.143:0/829104847 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3000b980 con 0x7f1d4006f200 2023-09-29T23:29:48.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d3effd700 1 -- 172.21.15.143:0/829104847 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000bd20 con 0x7f1d4006f200 2023-09-29T23:29:48.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12f406f060 0x7f12f4153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409f520 0x7f12f40a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12f409fb40 0x7f12f40a7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40a7960 con 0x7f12f406f060 2023-09-29T23:29:48.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40a1fc0 con 0x7f12f409fb40 2023-09-29T23:29:48.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f40a6990 con 0x7f12f409f520 2023-09-29T23:29:48.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f8e5d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12f406f060 0x7f12f4153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f8e5d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12f406f060 0x7f12f4153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34622/0 (socket says 172.21.15.143:34622) 2023-09-29T23:29:48.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f8e5d700 1 -- 172.21.15.143:0/1239489702 learned_addr learned my addr 172.21.15.143:0/1239489702 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:48.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f3fff700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409f520 0x7f12f40a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f965e700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12f409fb40 0x7f12f40a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f8e5d700 1 -- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409f520 msgr2=0x7f12f40a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f8e5d700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409f520 0x7f12f40a14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f8e5d700 1 -- 172.21.15.143:0/1239489702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12f409fb40 msgr2=0x7f12f40a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f8e5d700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12f409fb40 0x7f12f40a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f8e5d700 1 -- 172.21.15.143:0/1239489702 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e4009420 con 0x7f12f406f060 2023-09-29T23:29:48.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f3fff700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409f520 0x7f12f40a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:48.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 -- 172.21.15.143:0/829104847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4006f200 msgr2=0x7f1d400b0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 --2- 172.21.15.143:0/829104847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4006f200 0x7f1d400b0590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1d40068e60 tx=0x7f1d30009250 comp rx=0 tx=0).stop 2023-09-29T23:29:48.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 -- 172.21.15.143:0/829104847 shutdown_connections 2023-09-29T23:29:48.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 --2- 172.21.15.143:0/829104847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4006f200 0x7f1d400b0590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 --2- 172.21.15.143:0/829104847 >> v2:172.21.15.156:3300/0 conn(0x7f1d4006ee30 0x7f1d400a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f1d468c3700 1 -- 172.21.15.143:0/829104847 >> 172.21.15.143:0/829104847 conn(0x7f1d400b2ee0 msgr2=0x7f1d400b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f8e5d700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12f406f060 0x7f12f4153570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00b610 tx=0x7f12ec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f0ff9700 1 -- 172.21.15.143:0/1239489702 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0129e0 con 0x7f12f406f060 2023-09-29T23:29:48.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f0ff9700 1 -- 172.21.15.143:0/1239489702 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec012b60 con 0x7f12f406f060 2023-09-29T23:29:48.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f0ff9700 1 -- 172.21.15.143:0/1239489702 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec027470 con 0x7f12f406f060 2023-09-29T23:29:48.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 -- 172.21.15.143:0/1239489702 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ec009a20 con 0x7f12f406f060 2023-09-29T23:29:48.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.532+0000 7f12f9e5f700 1 -- 172.21.15.143:0/1239489702 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f4153ea0 con 0x7f12f406f060 2023-09-29T23:29:48.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d468c3700 1 -- 172.21.15.143:0/829104847 shutdown_connections 2023-09-29T23:29:48.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f12f9e5f700 1 -- 172.21.15.143:0/1239489702 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d00050b0 con 0x7f12f406f060 2023-09-29T23:29:48.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d468c3700 1 -- 172.21.15.143:0/829104847 wait complete. 2023-09-29T23:29:48.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f12f0ff9700 1 -- 172.21.15.143:0/1239489702 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f12ec024070 con 0x7f12f406f060 2023-09-29T23:29:48.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f12f0ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d468c3700 1 Processor -- start 2023-09-29T23:29:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d468c3700 1 -- start start 2023-09-29T23:29:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d468c3700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d4006ee30 0x7f1d4009fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d468c3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d4006f200 0x7f1d40148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d468c3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4009c650 0x7f1d4006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d468c3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40149220 con 0x7f1d4009c650 2023-09-29T23:29:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d468c3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40149450 con 0x7f1d4006f200 2023-09-29T23:29:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d468c3700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401496e0 con 0x7f1d4006ee30 2023-09-29T23:29:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f12f0ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f12f0ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f12f0ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f12f0ff9700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12d808e880 0x7f12d8090d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d44e60700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4009c650 0x7f1d4006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d44e60700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4009c650 0x7f1d4006c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34632/0 (socket says 172.21.15.143:34632) 2023-09-29T23:29:48.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d44e60700 1 -- 172.21.15.143:0/864336227 learned_addr learned my addr 172.21.15.143:0/864336227 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:48.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d3ffff700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d4006ee30 0x7f1d4009fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f12f0ff9700 1 -- 172.21.15.143:0/1239489702 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f12ec0b6a20 con 0x7f12f406f060 2023-09-29T23:29:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f12f3fff700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12d808e880 0x7f12d8090d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d3f7fe700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d4006f200 0x7f1d40148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d44e60700 1 -- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d4006ee30 msgr2=0x7f1d4009fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d44e60700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d4006ee30 0x7f1d4009fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d44e60700 1 -- 172.21.15.143:0/864336227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d4006f200 msgr2=0x7f1d40148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d44e60700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d4006f200 0x7f1d40148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d44e60700 1 -- 172.21.15.143:0/864336227 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d300094f0 con 0x7f1d4009c650 2023-09-29T23:29:48.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d3f7fe700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d4006f200 0x7f1d40148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-29T23:29:48.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f1d3ffff700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d4006ee30 0x7f1d4009fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:48.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.536+0000 7f12f3fff700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12d808e880 0x7f12d8090d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12e8007fd0 tx=0x7f12e8001aa0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.540+0000 7f1d44e60700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4009c650 0x7f1d4006c8d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800bc30 tx=0x7f1d38009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.540+0000 7f1d37fff700 1 -- 172.21.15.143:0/864336227 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800c900 con 0x7f1d4009c650 2023-09-29T23:29:48.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.540+0000 7f1d37fff700 1 -- 172.21.15.143:0/864336227 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3800caa0 con 0x7f1d4009c650 2023-09-29T23:29:48.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.540+0000 7f1d37fff700 1 -- 172.21.15.143:0/864336227 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800ce60 con 0x7f1d4009c650 2023-09-29T23:29:48.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.540+0000 7f1d468c3700 1 -- 172.21.15.143:0/864336227 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d400a0480 con 0x7f1d4009c650 2023-09-29T23:29:48.549 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.540+0000 7f12f0ff9700 1 -- 172.21.15.143:0/1239489702 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f12ec0874e0 con 0x7f12f406f060 2023-09-29T23:29:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.540+0000 7f1d468c3700 1 -- 172.21.15.143:0/864336227 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4006e1b0 con 0x7f1d4009c650 2023-09-29T23:29:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.544+0000 7f1d37fff700 1 -- 172.21.15.143:0/864336227 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f1d38014620 con 0x7f1d4009c650 2023-09-29T23:29:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.544+0000 7f1d37fff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.544+0000 7f1d37fff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:48.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.544+0000 7f1d37fff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.544+0000 7f1d37fff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.544+0000 7f1d37fff700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1d2008e8b0 0x7f1d20090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.544+0000 7f1d3ffff700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1d2008e8b0 0x7f1d20090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.544+0000 7f1d37fff700 1 -- 172.21.15.143:0/864336227 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f1d380b4800 con 0x7f1d4009c650 2023-09-29T23:29:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.544+0000 7f1d468c3700 1 -- 172.21.15.143:0/864336227 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d4006e3a0 con 0x7f1d4009c650 2023-09-29T23:29:48.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.544+0000 7f1d3ffff700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1d2008e8b0 0x7f1d20090d60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1d28006fd0 tx=0x7f1d28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:48.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.552+0000 7f1d37fff700 1 -- 172.21.15.143:0/864336227 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1d3804abe0 con 0x7f1d4009c650 2023-09-29T23:29:48.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.688+0000 7f39db3df700 1 -- 172.21.15.143:0/1755262302 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f39d40aebd0 con 0x7f39d406da20 2023-09-29T23:29:48.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.688+0000 7f39c97fa700 1 -- 172.21.15.143:0/1755262302 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f39d0021090 con 0x7f39d406da20 2023-09-29T23:29:48.691 INFO:teuthology.orchestra.run.smithi143.stdout:38654705670 2023-09-29T23:29:48.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 10 mgrc shutdown 2023-09-29T23:29:48.693 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 -- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39b808e8e0 msgr2=0x7f39b8090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39b808e8e0 0x7f39b8090d90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39d404c030 tx=0x7f39c4017000 comp rx=0 tx=0).stop 2023-09-29T23:29:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 -- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d406da20 msgr2=0x7f39d411b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d406da20 0x7f39d411b400 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f39d000ec80 tx=0x7f39d00091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 -- 172.21.15.143:0/1755262302 shutdown_connections 2023-09-29T23:29:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39b808e8e0 0x7f39b8090d90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f39d40b3be0 0x7f39d413cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39d40b3810 0x7f39d4147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 --2- 172.21.15.143:0/1755262302 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39d406da20 0x7f39d411b400 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 -- 172.21.15.143:0/1755262302 >> 172.21.15.143:0/1755262302 conn(0x7f39d4069820 msgr2=0x7f39d40b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 -- 172.21.15.143:0/1755262302 shutdown_connections 2023-09-29T23:29:48.694 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.692+0000 7f39db3df700 1 -- 172.21.15.143:0/1755262302 wait complete. 2023-09-29T23:29:48.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705670 for osd.0 2023-09-29T23:29:48.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 -- 172.21.15.143:0/864336227 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1d4005ecb0 con 0x7f1d4009c650 2023-09-29T23:29:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d37fff700 1 -- 172.21.15.143:0/864336227 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1d380852c0 con 0x7f1d4009c650 2023-09-29T23:29:48.718 INFO:teuthology.orchestra.run.smithi143.stdout:38654705669 2023-09-29T23:29:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 10 mgrc shutdown 2023-09-29T23:29:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 -- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1d2008e8b0 msgr2=0x7f1d20090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1d2008e8b0 0x7f1d20090d60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1d28006fd0 tx=0x7f1d28008040 comp rx=0 tx=0).stop 2023-09-29T23:29:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 -- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4009c650 msgr2=0x7f1d4006c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4009c650 0x7f1d4006c8d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800bc30 tx=0x7f1d38009820 comp rx=0 tx=0).stop 2023-09-29T23:29:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 -- 172.21.15.143:0/864336227 shutdown_connections 2023-09-29T23:29:48.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1d2008e8b0 0x7f1d20090d60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1d4009c650 0x7f1d4006c8d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d4006f200 0x7f1d40148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 --2- 172.21.15.143:0/864336227 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f1d4006ee30 0x7f1d4009fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 -- 172.21.15.143:0/864336227 >> 172.21.15.143:0/864336227 conn(0x7f1d400b2ee0 msgr2=0x7f1d40146e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 -- 172.21.15.143:0/864336227 shutdown_connections 2023-09-29T23:29:48.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.716+0000 7f1d468c3700 1 -- 172.21.15.143:0/864336227 wait complete. 2023-09-29T23:29:48.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.728+0000 7f2b80f1c700 1 -- 172.21.15.143:0/759353236 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2b7c0a2600 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.730 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.728+0000 7f2b637fe700 1 -- 172.21.15.143:0/759353236 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b74009900 con 0x7f2b7c0a5e20 2023-09-29T23:29:48.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705669 for osd.2 2023-09-29T23:29:48.731 INFO:teuthology.orchestra.run.smithi143.stdout:38654705669 2023-09-29T23:29:48.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 10 mgrc shutdown 2023-09-29T23:29:48.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 -- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2b6808e8b0 msgr2=0x7f2b68090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2b6808e8b0 0x7f2b68090d60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c0058d0 tx=0x7f2b6c007ac0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 -- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c0a5e20 msgr2=0x7f2b7c09b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c0a5e20 0x7f2b7c09b570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2b74007f50 tx=0x7f2b740097e0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 -- 172.21.15.143:0/759353236 shutdown_connections 2023-09-29T23:29:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2b6808e8b0 0x7f2b68090d60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2b7c1436e0 0x7f2b7c153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b7c135f80 0x7f2b7c06df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 --2- 172.21.15.143:0/759353236 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2b7c0a5e20 0x7f2b7c09b570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 -- 172.21.15.143:0/759353236 >> 172.21.15.143:0/759353236 conn(0x7f2b7c067da0 msgr2=0x7f2b7c09f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 -- 172.21.15.143:0/759353236 shutdown_connections 2023-09-29T23:29:48.734 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.732+0000 7f2b80f1c700 1 -- 172.21.15.143:0/759353236 wait complete. 2023-09-29T23:29:48.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705669 for osd.1 2023-09-29T23:29:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.748+0000 7f12f9e5f700 1 -- 172.21.15.143:0/1239489702 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f12d0005630 con 0x7f12f406f060 2023-09-29T23:29:48.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.748+0000 7f12f0ff9700 1 -- 172.21.15.143:0/1239489702 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f12ec02c020 con 0x7f12f406f060 2023-09-29T23:29:48.751 INFO:teuthology.orchestra.run.smithi143.stdout:38654705670 2023-09-29T23:29:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 10 mgrc shutdown 2023-09-29T23:29:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 -- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12d808e880 msgr2=0x7f12d8090d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12d808e880 0x7f12d8090d30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12e8007fd0 tx=0x7f12e8001aa0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 -- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12f406f060 msgr2=0x7f12f4153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12f406f060 0x7f12f4153570 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f12ec00b610 tx=0x7f12ec018000 comp rx=0 tx=0).stop 2023-09-29T23:29:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 -- 172.21.15.143:0/1239489702 shutdown_connections 2023-09-29T23:29:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12d808e880 0x7f12d8090d30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12f409fb40 0x7f12f40a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12f409f520 0x7f12f40a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 --2- 172.21.15.143:0/1239489702 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12f406f060 0x7f12f4153570 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:48.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 -- 172.21.15.143:0/1239489702 >> 172.21.15.143:0/1239489702 conn(0x7f12f40b2c70 msgr2=0x7f12f413f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:48.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 -- 172.21.15.143:0/1239489702 shutdown_connections 2023-09-29T23:29:48.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:48.752+0000 7f12f9e5f700 1 -- 172.21.15.143:0/1239489702 wait complete. 2023-09-29T23:29:48.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705670 for osd.3 2023-09-29T23:29:49.704 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-29T23:29:49.731 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-29T23:29:49.747 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-29T23:29:49.765 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-29T23:29:49.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 Processor -- start 2023-09-29T23:29:49.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 -- start start 2023-09-29T23:29:49.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f38980a6660 0x7f389806aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f389806b3e0 0x7f389813f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3898140f20 con 0x7f389813fb80 2023-09-29T23:29:49.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f389813ffe0 con 0x7f38980a6660 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f389806a980 con 0x7f389806b3e0 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3896ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f38980a6660 0x7f389806aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3896ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f38980a6660 0x7f389806aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34634/0 (socket says 172.21.15.143:34634) 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f38977fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f389813fb80 0x7f3898069900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60716/0 (socket says 172.21.15.143:60716) 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3896ffd700 1 -- 172.21.15.143:0/2651970977 learned_addr learned my addr 172.21.15.143:0/2651970977 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f38967fc700 1 --2- 172.21.15.143:0/2651970977 >> v2:172.21.15.156:3300/0 conn(0x7f389806b3e0 0x7f389813f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3896ffd700 1 -- 172.21.15.143:0/2651970977 >> v2:172.21.15.156:3300/0 conn(0x7f389806b3e0 msgr2=0x7f389813f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3895ffb700 1 -- 172.21.15.143:0/2651970977 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936117411 0 0) 0x7f3898140f20 con 0x7f389813fb80 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3896ffd700 1 --2- 172.21.15.143:0/2651970977 >> v2:172.21.15.156:3300/0 conn(0x7f389806b3e0 0x7f389813f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3896ffd700 1 -- 172.21.15.143:0/2651970977 >> v1:172.21.15.143:6789/0 conn(0x7f389813fb80 legacy=0x7f3898069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3896ffd700 1 -- 172.21.15.143:0/2651970977 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38981401c0 con 0x7f38980a6660 2023-09-29T23:29:49.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3896ffd700 1 --2- 172.21.15.143:0/2651970977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38980a6660 0x7f389806aea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f38980b4170 tx=0x7f3888009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36566c4aee6669b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3895ffb700 1 -- 172.21.15.143:0/2651970977 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888009e70 con 0x7f38980a6660 2023-09-29T23:29:49.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3895ffb700 1 -- 172.21.15.143:0/2651970977 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388800b860 con 0x7f38980a6660 2023-09-29T23:29:49.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f3895ffb700 1 -- 172.21.15.143:0/2651970977 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388800bc00 con 0x7f38980a6660 2023-09-29T23:29:49.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 -- 172.21.15.143:0/2651970977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38980a6660 msgr2=0x7f389806aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 --2- 172.21.15.143:0/2651970977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38980a6660 0x7f389806aea0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f38980b4170 tx=0x7f3888009250 comp rx=0 tx=0).stop 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 -- 172.21.15.143:0/2651970977 shutdown_connections 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 --2- 172.21.15.143:0/2651970977 >> v2:172.21.15.156:3300/0 conn(0x7f389806b3e0 0x7f389813f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 --2- 172.21.15.143:0/2651970977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38980a6660 0x7f389806aea0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 -- 172.21.15.143:0/2651970977 >> 172.21.15.143:0/2651970977 conn(0x7f389806be10 msgr2=0x7f389806c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 -- 172.21.15.143:0/2651970977 shutdown_connections 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.832+0000 7f389d83a700 1 -- 172.21.15.143:0/2651970977 wait complete. 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f389d83a700 1 Processor -- start 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f389d83a700 1 -- start start 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f389d83a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f389806b3e0 0x7f38980b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f389d83a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38980a6660 0x7f38980ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f389d83a700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f389813fb80 0x7f38980aea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f389d83a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38980ae0e0 con 0x7f389806b3e0 2023-09-29T23:29:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f389d83a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38980aef80 con 0x7f38980a6660 2023-09-29T23:29:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f389d83a700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38980af190 con 0x7f389813fb80 2023-09-29T23:29:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f3896ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f389806b3e0 0x7f38980b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f38977fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f389813fb80 0x7f38980aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f38977fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f389813fb80 0x7f38980aea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60496/0 (socket says 172.21.15.143:60496) 2023-09-29T23:29:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f3896ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f389806b3e0 0x7f38980b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34640/0 (socket says 172.21.15.143:34640) 2023-09-29T23:29:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f38977fe700 1 -- 172.21.15.143:0/416358897 learned_addr learned my addr 172.21.15.143:0/416358897 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:49.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f38967fc700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38980a6660 0x7f38980ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f38977fe700 1 -- 172.21.15.143:0/416358897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38980a6660 msgr2=0x7f38980ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f38977fe700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38980a6660 0x7f38980ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f38977fe700 1 -- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f389806b3e0 msgr2=0x7f38980b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f38977fe700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f389806b3e0 0x7f38980b31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f38977fe700 1 -- 172.21.15.143:0/416358897 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38880094f0 con 0x7f389813fb80 2023-09-29T23:29:49.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f3896ffd700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f389806b3e0 0x7f38980b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:49.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f38977fe700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f389813fb80 0x7f38980aea40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f389000ccc0 tx=0x7f3890001a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f387effd700 1 -- 172.21.15.143:0/416358897 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890005df0 con 0x7f389813fb80 2023-09-29T23:29:49.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f387effd700 1 -- 172.21.15.143:0/416358897 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389000a600 con 0x7f389813fb80 2023-09-29T23:29:49.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f389d83a700 1 -- 172.21.15.143:0/416358897 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38980b0770 con 0x7f389813fb80 2023-09-29T23:29:49.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f387effd700 1 -- 172.21.15.143:0/416358897 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389000a9c0 con 0x7f389813fb80 2023-09-29T23:29:49.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.836+0000 7f389d83a700 1 -- 172.21.15.143:0/416358897 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389813e5a0 con 0x7f389813fb80 2023-09-29T23:29:49.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.840+0000 7f387effd700 1 -- 172.21.15.143:0/416358897 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f38900070f0 con 0x7f389813fb80 2023-09-29T23:29:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.840+0000 7f387effd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.840+0000 7f387effd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.840+0000 7f387effd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.840+0000 7f387effd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:49.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.840+0000 7f387effd700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f388408e8b0 0x7f3884090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.840+0000 7f3896ffd700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f388408e8b0 0x7f3884090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.840+0000 7f387effd700 1 -- 172.21.15.143:0/416358897 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f38900b4820 con 0x7f389813fb80 2023-09-29T23:29:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.840+0000 7f3896ffd700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f388408e8b0 0x7f3884090d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38980b4170 tx=0x7f3888017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.840+0000 7f389d83a700 1 -- 172.21.15.143:0/416358897 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38980ae220 con 0x7f389813fb80 2023-09-29T23:29:49.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.852+0000 7f387effd700 1 -- 172.21.15.143:0/416358897 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f38980ae220 con 0x7f389813fb80 2023-09-29T23:29:49.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.872+0000 7efcfe60f700 1 Processor -- start 2023-09-29T23:29:49.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.872+0000 7efcfe60f700 1 -- start start 2023-09-29T23:29:49.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfe60f700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efcf809e9d0 0x7efcf8148910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfe60f700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf806f430 con 0x7efcf809f820 2023-09-29T23:29:49.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfe60f700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcf809d8b0 con 0x7efcf806f060 2023-09-29T23:29:49.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfe60f700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf809db70 con 0x7efcf809e9d0 2023-09-29T23:29:49.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfcbac700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efcf809e9d0 0x7efcf8148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfcbac700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efcf809e9d0 0x7efcf8148910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34654/0 (socket says 172.21.15.143:34654) 2023-09-29T23:29:49.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfcbac700 1 -- 172.21.15.143:0/4152473404 learned_addr learned my addr 172.21.15.143:0/4152473404 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:49.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcf77fe700 1 --1- 172.21.15.143:0/4152473404 >> v1:172.21.15.143:6789/0 conn(0x7efcf809f820 0x7efcf8140cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60732/0 (socket says 172.21.15.143:60732) 2023-09-29T23:29:49.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfcbac700 1 -- 172.21.15.143:0/4152473404 >> v1:172.21.15.156:6789/0 conn(0x7efcf806f060 legacy=0x7efcf809ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfcbac700 1 -- 172.21.15.143:0/4152473404 >> v1:172.21.15.143:6789/0 conn(0x7efcf809f820 legacy=0x7efcf8140cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcf6ffd700 1 -- 172.21.15.143:0/4152473404 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2677012084 0 0) 0x7efcf806f430 con 0x7efcf809f820 2023-09-29T23:29:49.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfcbac700 1 -- 172.21.15.143:0/4152473404 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf809c990 con 0x7efcf809e9d0 2023-09-29T23:29:49.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfcbac700 1 --2- 172.21.15.143:0/4152473404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcf809e9d0 0x7efcf8148910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efcf000d1a0 tx=0x7efcf000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79caf9ca92809935 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.880 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcf6ffd700 1 -- 172.21.15.143:0/4152473404 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0010070 con 0x7efcf809e9d0 2023-09-29T23:29:49.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcf6ffd700 1 -- 172.21.15.143:0/4152473404 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf000b0d0 con 0x7efcf809e9d0 2023-09-29T23:29:49.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcf6ffd700 1 -- 172.21.15.143:0/4152473404 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf000b490 con 0x7efcf809e9d0 2023-09-29T23:29:49.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfe60f700 1 -- 172.21.15.143:0/4152473404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcf809e9d0 msgr2=0x7efcf8148910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfe60f700 1 --2- 172.21.15.143:0/4152473404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcf809e9d0 0x7efcf8148910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7efcf000d1a0 tx=0x7efcf000ba50 comp rx=0 tx=0).stop 2023-09-29T23:29:49.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfe60f700 1 -- 172.21.15.143:0/4152473404 shutdown_connections 2023-09-29T23:29:49.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfe60f700 1 --2- 172.21.15.143:0/4152473404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcf809e9d0 0x7efcf8148910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfe60f700 1 -- 172.21.15.143:0/4152473404 >> 172.21.15.143:0/4152473404 conn(0x7efcf80b2ee0 msgr2=0x7efcf80b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:49.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfe60f700 1 -- 172.21.15.143:0/4152473404 shutdown_connections 2023-09-29T23:29:49.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.876+0000 7efcfe60f700 1 -- 172.21.15.143:0/4152473404 wait complete. 2023-09-29T23:29:49.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfe60f700 1 Processor -- start 2023-09-29T23:29:49.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfe60f700 1 -- start start 2023-09-29T23:29:49.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfe60f700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efcf806f060 0x7efcf806ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfe60f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcf809e9d0 0x7efcf8143000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfe60f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcf809f820 0x7efcf8152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfe60f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf80a6ae0 con 0x7efcf809f820 2023-09-29T23:29:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfe60f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf806d7b0 con 0x7efcf809e9d0 2023-09-29T23:29:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfe60f700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf81536c0 con 0x7efcf806f060 2023-09-29T23:29:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcf7fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efcf806f060 0x7efcf806ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfcbac700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcf809f820 0x7efcf8152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfcbac700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcf809f820 0x7efcf8152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34656/0 (socket says 172.21.15.143:34656) 2023-09-29T23:29:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfcbac700 1 -- 172.21.15.143:0/2724002504 learned_addr learned my addr 172.21.15.143:0/2724002504 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcf77fe700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcf809e9d0 0x7efcf8143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfcbac700 1 -- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efcf806f060 msgr2=0x7efcf806ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfcbac700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efcf806f060 0x7efcf806ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfcbac700 1 -- 172.21.15.143:0/2724002504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcf809e9d0 msgr2=0x7efcf8143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfcbac700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcf809e9d0 0x7efcf8143000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfcbac700 1 -- 172.21.15.143:0/2724002504 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf000bcf0 con 0x7efcf809f820 2023-09-29T23:29:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfcbac700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcf809f820 0x7efcf8152ff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efcf000b660 tx=0x7efcf001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcdffff700 1 -- 172.21.15.143:0/2724002504 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf000ad50 con 0x7efcf809f820 2023-09-29T23:29:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcdffff700 1 -- 172.21.15.143:0/2724002504 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf001e440 con 0x7efcf809f820 2023-09-29T23:29:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfe60f700 1 -- 172.21.15.143:0/2724002504 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf8153980 con 0x7efcf809f820 2023-09-29T23:29:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcdffff700 1 -- 172.21.15.143:0/2724002504 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf001e7e0 con 0x7efcf809f820 2023-09-29T23:29:49.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfe60f700 1 -- 172.21.15.143:0/2724002504 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf806e470 con 0x7efcf809f820 2023-09-29T23:29:49.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcfe60f700 1 -- 172.21.15.143:0/2724002504 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf805ecb0 con 0x7efcf809f820 2023-09-29T23:29:49.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcdffff700 1 -- 172.21.15.143:0/2724002504 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7efcf0003040 con 0x7efcf809f820 2023-09-29T23:29:49.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.880+0000 7efcdffff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:49.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.884+0000 7efcdffff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:49.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.884+0000 7efcdffff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:49.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.884+0000 7efcdffff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:49.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.884+0000 7efcdffff700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efce408e8e0 0x7efce4090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.884+0000 7efcf7fff700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efce408e8e0 0x7efce4090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.884+0000 7efcdffff700 1 -- 172.21.15.143:0/2724002504 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7efcf00b6540 con 0x7efcf809f820 2023-09-29T23:29:49.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.884+0000 7efcf7fff700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efce408e8e0 0x7efce4090d90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efce8003a80 tx=0x7efce8007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70fc730700 1 Processor -- start 2023-09-29T23:29:49.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70fc730700 1 -- start start 2023-09-29T23:29:49.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70fc730700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f70f40a78b0 0x7f70f40a7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70fc730700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f40a68d0 con 0x7f70f40b2f10 2023-09-29T23:29:49.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70fc730700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f40a4d00 con 0x7f70f40b32e0 2023-09-29T23:29:49.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70fc730700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f40a7420 con 0x7f70f40a78b0 2023-09-29T23:29:49.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70fa4cc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f70f40b2f10 0x7f70f40a61c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:60742/0 (socket says 172.21.15.143:60742) 2023-09-29T23:29:49.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70fa4cc700 1 -- 172.21.15.143:0/2181045308 learned_addr learned my addr 172.21.15.143:0/2181045308 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70faccd700 1 --2- 172.21.15.143:0/2181045308 >> v2:172.21.15.156:3300/0 conn(0x7f70f40a78b0 0x7f70f40a7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70f94ca700 1 -- 172.21.15.143:0/2181045308 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767116589 0 0) 0x7f70f40a4d00 con 0x7f70f40b32e0 2023-09-29T23:29:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70f94ca700 1 -- 172.21.15.143:0/2181045308 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70e0003570 con 0x7f70f40b32e0 2023-09-29T23:29:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70faccd700 1 -- 172.21.15.143:0/2181045308 >> v1:172.21.15.156:6789/0 conn(0x7f70f40b32e0 legacy=0x7f70f413d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70faccd700 1 -- 172.21.15.143:0/2181045308 >> v1:172.21.15.143:6789/0 conn(0x7f70f40b2f10 legacy=0x7f70f40a61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70faccd700 1 -- 172.21.15.143:0/2181045308 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f41415e0 con 0x7f70f40a78b0 2023-09-29T23:29:49.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70faccd700 1 --2- 172.21.15.143:0/2181045308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40a78b0 0x7f70f40a7d50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009e60 tx=0x7f70ec009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ea43674435670cb server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70f94ca700 1 -- 172.21.15.143:0/2181045308 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec00e070 con 0x7f70f40a78b0 2023-09-29T23:29:49.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70f94ca700 1 -- 172.21.15.143:0/2181045308 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec00bbf0 con 0x7f70f40a78b0 2023-09-29T23:29:49.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70f94ca700 1 -- 172.21.15.143:0/2181045308 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec012430 con 0x7f70f40a78b0 2023-09-29T23:29:49.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7efcdffff700 1 -- 172.21.15.143:0/2724002504 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efcf0087000 con 0x7efcf809f820 2023-09-29T23:29:49.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70fc730700 1 -- 172.21.15.143:0/2181045308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40a78b0 msgr2=0x7f70f40a7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.888+0000 7f70fc730700 1 --2- 172.21.15.143:0/2181045308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40a78b0 0x7f70f40a7d50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009e60 tx=0x7f70ec009780 comp rx=0 tx=0).stop 2023-09-29T23:29:49.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.892+0000 7f70fc730700 1 -- 172.21.15.143:0/2181045308 shutdown_connections 2023-09-29T23:29:49.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.892+0000 7f70fc730700 1 --2- 172.21.15.143:0/2181045308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40a78b0 0x7f70f40a7d50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.892+0000 7f70fc730700 1 -- 172.21.15.143:0/2181045308 >> 172.21.15.143:0/2181045308 conn(0x7f70f4067dc0 msgr2=0x7f70f40b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.892+0000 7faf7efdf700 1 Processor -- start 2023-09-29T23:29:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.892+0000 7faf7efdf700 1 -- start start 2023-09-29T23:29:49.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.892+0000 7f70fc730700 1 -- 172.21.15.143:0/2181045308 shutdown_connections 2023-09-29T23:29:49.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.892+0000 7f70fc730700 1 -- 172.21.15.143:0/2181045308 wait complete. 2023-09-29T23:29:49.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.892+0000 7faf7efdf700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faf780a0e50 0x7faf780a1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.892+0000 7faf7efdf700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faf78148a40 0x7faf7814ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.892+0000 7faf7efdf700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf780b31f0 con 0x7faf780b1940 2023-09-29T23:29:49.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7efdf700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf780b1d10 con 0x7faf78148a40 2023-09-29T23:29:49.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7efdf700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf780a04a0 con 0x7faf780a0e50 2023-09-29T23:29:49.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7cd7b700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7faf780b1940 0x7faf7809f920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.143:47878/0 (socket says 172.21.15.143:47878) 2023-09-29T23:29:49.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7cd7b700 1 -- 172.21.15.143:0/1915931273 learned_addr learned my addr 172.21.15.143:0/1915931273 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:29:49.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7d57c700 1 --2- 172.21.15.143:0/1915931273 >> v2:172.21.15.143:3300/0 conn(0x7faf78148a40 0x7faf7814ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70fc730700 1 Processor -- start 2023-09-29T23:29:49.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf77fff700 1 --2- 172.21.15.143:0/1915931273 >> v2:172.21.15.156:3300/0 conn(0x7faf780a0e50 0x7faf780a1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70fc730700 1 -- start start 2023-09-29T23:29:49.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7d57c700 1 -- 172.21.15.143:0/1915931273 >> v2:172.21.15.156:3300/0 conn(0x7faf780a0e50 msgr2=0x7faf780a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:49.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7d57c700 1 --2- 172.21.15.143:0/1915931273 >> v2:172.21.15.156:3300/0 conn(0x7faf780a0e50 0x7faf780a1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7d57c700 1 -- 172.21.15.143:0/1915931273 >> v1:172.21.15.156:6789/0 conn(0x7faf780b1940 legacy=0x7faf7809f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7d57c700 1 -- 172.21.15.143:0/1915931273 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf78140410 con 0x7faf78148a40 2023-09-29T23:29:49.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7d57c700 1 --2- 172.21.15.143:0/1915931273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf78148a40 0x7faf7814ae20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faf78068250 tx=0x7faf6c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfe2d12888ad365 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf777fe700 1 -- 172.21.15.143:0/1915931273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c011630 con 0x7faf78148a40 2023-09-29T23:29:49.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70fc730700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40a78b0 0x7f70f413c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70fc730700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f70f40b2f10 0x7f70f4137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70fc730700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70f40b32e0 0x7f70f40ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70fc730700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40ac760 con 0x7f70f40b32e0 2023-09-29T23:29:49.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70fc730700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f414de70 con 0x7f70f40a78b0 2023-09-29T23:29:49.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70fc730700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f40ac0a0 con 0x7f70f40b2f10 2023-09-29T23:29:49.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70faccd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70f40b32e0 0x7f70f40ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70faccd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70f40b32e0 0x7f70f40ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34678/0 (socket says 172.21.15.143:34678) 2023-09-29T23:29:49.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70faccd700 1 -- 172.21.15.143:0/2495579668 learned_addr learned my addr 172.21.15.143:0/2495579668 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:49.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70fa4cc700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40a78b0 0x7f70f413c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70f9ccb700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f70f40b2f10 0x7f70f4137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70faccd700 1 -- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f70f40b2f10 msgr2=0x7f70f4137bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70faccd700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f70f40b2f10 0x7f70f4137bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70faccd700 1 -- 172.21.15.143:0/2495579668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40a78b0 msgr2=0x7f70f413c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70faccd700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40a78b0 0x7f70f413c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70faccd700 1 -- 172.21.15.143:0/2495579668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec009a20 con 0x7f70f40b32e0 2023-09-29T23:29:49.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf777fe700 1 -- 172.21.15.143:0/1915931273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c011790 con 0x7faf78148a40 2023-09-29T23:29:49.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf777fe700 1 -- 172.21.15.143:0/1915931273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c011b30 con 0x7faf78148a40 2023-09-29T23:29:49.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7efdf700 1 -- 172.21.15.143:0/1915931273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf78148a40 msgr2=0x7faf7814ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7faf7efdf700 1 --2- 172.21.15.143:0/1915931273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf78148a40 0x7faf7814ae20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7faf78068250 tx=0x7faf6c0091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70faccd700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70f40b32e0 0x7f70f40ace60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f70f406ea40 tx=0x7f70ec019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70ea7fc700 1 -- 172.21.15.143:0/2495579668 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec027470 con 0x7f70f40b32e0 2023-09-29T23:29:49.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70ea7fc700 1 -- 172.21.15.143:0/2495579668 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec0275f0 con 0x7f70f40b32e0 2023-09-29T23:29:49.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70fc730700 1 -- 172.21.15.143:0/2495579668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f4138100 con 0x7f70f40b32e0 2023-09-29T23:29:49.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70ea7fc700 1 -- 172.21.15.143:0/2495579668 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec0279b0 con 0x7f70f40b32e0 2023-09-29T23:29:49.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.896+0000 7f70fc730700 1 -- 172.21.15.143:0/2495579668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f40ad5c0 con 0x7f70f40b32e0 2023-09-29T23:29:49.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7faf7efdf700 1 -- 172.21.15.143:0/1915931273 shutdown_connections 2023-09-29T23:29:49.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7faf7efdf700 1 --2- 172.21.15.143:0/1915931273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf78148a40 0x7faf7814ae20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7faf7efdf700 1 --2- 172.21.15.143:0/1915931273 >> v2:172.21.15.156:3300/0 conn(0x7faf780a0e50 0x7faf780a1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7faf7efdf700 1 -- 172.21.15.143:0/1915931273 >> 172.21.15.143:0/1915931273 conn(0x7faf78068660 msgr2=0x7faf78069ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:49.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7f70ea7fc700 1 -- 172.21.15.143:0/2495579668 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f70ec024070 con 0x7f70f40b32e0 2023-09-29T23:29:49.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7f70ea7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:49.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7faf7efdf700 1 -- 172.21.15.143:0/1915931273 shutdown_connections 2023-09-29T23:29:49.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7faf7efdf700 1 -- 172.21.15.143:0/1915931273 wait complete. 2023-09-29T23:29:49.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7f70ea7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:49.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7f70ea7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:49.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7f70ea7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:49.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7f70ea7fc700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f70d808e800 0x7f70d8090cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7f70fa4cc700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f70d808e800 0x7f70d8090cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7f70ea7fc700 1 -- 172.21.15.143:0/2495579668 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f70ec0b8790 con 0x7f70f40b32e0 2023-09-29T23:29:49.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7f70fa4cc700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f70d808e800 0x7f70d8090cb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70e4007130 tx=0x7f70e4007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7faf7efdf700 1 Processor -- start 2023-09-29T23:29:49.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.900+0000 7faf7efdf700 1 -- start start 2023-09-29T23:29:49.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7efdf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf780a0e50 0x7faf7806da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7efdf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf780b1940 0x7faf7813cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7efdf700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf78148a40 0x7faf78146060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7efdf700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7806d520 con 0x7faf780b1940 2023-09-29T23:29:49.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7efdf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf78146af0 con 0x7faf780a0e50 2023-09-29T23:29:49.908 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7efdf700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf78146d90 con 0x7faf78148a40 2023-09-29T23:29:49.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7f70fc730700 1 -- 172.21.15.143:0/2495579668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f4138370 con 0x7f70f40b32e0 2023-09-29T23:29:49.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf77fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf780b1940 0x7faf7813cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7d57c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf78148a40 0x7faf78146060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf77fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf780b1940 0x7faf7813cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34682/0 (socket says 172.21.15.143:34682) 2023-09-29T23:29:49.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf77fff700 1 -- 172.21.15.143:0/2706324732 learned_addr learned my addr 172.21.15.143:0/2706324732 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:49.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7cd7b700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf780a0e50 0x7faf7806da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7d57c700 1 -- 172.21.15.143:0/2706324732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf780a0e50 msgr2=0x7faf7806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7d57c700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf780a0e50 0x7faf7806da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7d57c700 1 -- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf780b1940 msgr2=0x7faf7813cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:49.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7d57c700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf780b1940 0x7faf7813cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:49.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7d57c700 1 -- 172.21.15.143:0/2706324732 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c009420 con 0x7faf78148a40 2023-09-29T23:29:49.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf77fff700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf780b1940 0x7faf7813cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:49.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7d57c700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf78148a40 0x7faf78146060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf6c016000 tx=0x7faf6c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf74ff9700 1 -- 172.21.15.143:0/2706324732 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c011b10 con 0x7faf78148a40 2023-09-29T23:29:49.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf74ff9700 1 -- 172.21.15.143:0/2706324732 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c0233f0 con 0x7faf78148a40 2023-09-29T23:29:49.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7efdf700 1 -- 172.21.15.143:0/2706324732 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7806e3b0 con 0x7faf78148a40 2023-09-29T23:29:49.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf74ff9700 1 -- 172.21.15.143:0/2706324732 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c023790 con 0x7faf78148a40 2023-09-29T23:29:49.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.904+0000 7faf7efdf700 1 -- 172.21.15.143:0/2706324732 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7809eba0 con 0x7faf78148a40 2023-09-29T23:29:49.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.908+0000 7faf74ff9700 1 -- 172.21.15.143:0/2706324732 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7faf6c0193b0 con 0x7faf78148a40 2023-09-29T23:29:49.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.908+0000 7faf74ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:49.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.908+0000 7faf74ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:49.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.908+0000 7faf74ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:49.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.908+0000 7faf74ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:49.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.908+0000 7faf74ff9700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf6008e950 0x7faf60090e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:49.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.908+0000 7faf7cd7b700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf6008e950 0x7faf60090e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:49.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.908+0000 7faf74ff9700 1 -- 172.21.15.143:0/2706324732 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7faf6c0b59a0 con 0x7faf78148a40 2023-09-29T23:29:49.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.908+0000 7faf7cd7b700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf6008e950 0x7faf60090e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faf68007130 tx=0x7faf68007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:49.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.912+0000 7f70ea7fc700 1 -- 172.21.15.143:0/2495579668 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f70ec0bc060 con 0x7f70f40b32e0 2023-09-29T23:29:49.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.912+0000 7faf7efdf700 1 -- 172.21.15.143:0/2706324732 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf540050b0 con 0x7faf78148a40 2023-09-29T23:29:49.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:49.916+0000 7faf74ff9700 1 -- 172.21.15.143:0/2706324732 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faf6c086460 con 0x7faf78148a40 2023-09-29T23:29:50.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.036+0000 7f389d83a700 1 -- 172.21.15.143:0/416358897 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f38980ae220 con 0x7f389813fb80 2023-09-29T23:29:50.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.036+0000 7f387effd700 1 -- 172.21.15.143:0/416358897 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f38980ae220 con 0x7f389813fb80 2023-09-29T23:29:50.038 INFO:teuthology.orchestra.run.smithi143.stdout:38654705672 2023-09-29T23:29:50.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 10 mgrc shutdown 2023-09-29T23:29:50.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 -- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f388408e8b0 msgr2=0x7f3884090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f388408e8b0 0x7f3884090d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f38980b4170 tx=0x7f3888017000 comp rx=0 tx=0).stop 2023-09-29T23:29:50.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 -- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f389813fb80 msgr2=0x7f38980aea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f389813fb80 0x7f38980aea40 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f389000ccc0 tx=0x7f3890001a70 comp rx=0 tx=0).stop 2023-09-29T23:29:50.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 -- 172.21.15.143:0/416358897 shutdown_connections 2023-09-29T23:29:50.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f388408e8b0 0x7f3884090d60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f389813fb80 0x7f38980aea40 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38980a6660 0x7f38980ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 --2- 172.21.15.143:0/416358897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f389806b3e0 0x7f38980b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 -- 172.21.15.143:0/416358897 >> 172.21.15.143:0/416358897 conn(0x7f389806be10 msgr2=0x7f38980a4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:50.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 -- 172.21.15.143:0/416358897 shutdown_connections 2023-09-29T23:29:50.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.040+0000 7f389d83a700 1 -- 172.21.15.143:0/416358897 wait complete. 2023-09-29T23:29:50.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705672 for osd.0 2023-09-29T23:29:50.053 DEBUG:teuthology.parallel:result is None 2023-09-29T23:29:50.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.076+0000 7f70fc730700 1 -- 172.21.15.143:0/2495579668 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f70f405ecb0 con 0x7f70f40b32e0 2023-09-29T23:29:50.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.076+0000 7faf7efdf700 1 -- 172.21.15.143:0/2706324732 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7faf54005630 con 0x7faf78148a40 2023-09-29T23:29:50.079 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.076+0000 7f70ea7fc700 1 -- 172.21.15.143:0/2495579668 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70ec087020 con 0x7f70f40b32e0 2023-09-29T23:29:50.080 INFO:teuthology.orchestra.run.smithi143.stdout:38654705672 2023-09-29T23:29:50.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.076+0000 7faf74ff9700 1 -- 172.21.15.143:0/2706324732 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7faf6c08c020 con 0x7faf78148a40 2023-09-29T23:29:50.080 INFO:teuthology.orchestra.run.smithi143.stdout:38654705671 2023-09-29T23:29:50.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 10 mgrc shutdown 2023-09-29T23:29:50.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 -- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f70d808e800 msgr2=0x7f70d8090cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f70d808e800 0x7f70d8090cb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70e4007130 tx=0x7f70e4007eb0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 -- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70f40b32e0 msgr2=0x7f70f40ace60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70f40b32e0 0x7f70f40ace60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f70f406ea40 tx=0x7f70ec019000 comp rx=0 tx=0).stop 2023-09-29T23:29:50.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 -- 172.21.15.143:0/2495579668 shutdown_connections 2023-09-29T23:29:50.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f70d808e800 0x7f70d8090cb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f70f40b32e0 0x7f70f40ace60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f70f40b2f10 0x7f70f4137bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 --2- 172.21.15.143:0/2495579668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70f40a78b0 0x7f70f413c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 -- 172.21.15.143:0/2495579668 >> 172.21.15.143:0/2495579668 conn(0x7f70f4067dc0 msgr2=0x7f70f413fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:50.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 -- 172.21.15.143:0/2495579668 shutdown_connections 2023-09-29T23:29:50.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7f70fc730700 1 -- 172.21.15.143:0/2495579668 wait complete. 2023-09-29T23:29:50.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 10 mgrc shutdown 2023-09-29T23:29:50.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 -- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf6008e950 msgr2=0x7faf60090e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf6008e950 0x7faf60090e00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faf68007130 tx=0x7faf68007eb0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 -- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf78148a40 msgr2=0x7faf78146060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf78148a40 0x7faf78146060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7faf6c016000 tx=0x7faf6c017000 comp rx=0 tx=0).stop 2023-09-29T23:29:50.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 -- 172.21.15.143:0/2706324732 shutdown_connections 2023-09-29T23:29:50.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf6008e950 0x7faf60090e00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf78148a40 0x7faf78146060 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf780b1940 0x7faf7813cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 --2- 172.21.15.143:0/2706324732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf780a0e50 0x7faf7806da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 -- 172.21.15.143:0/2706324732 >> 172.21.15.143:0/2706324732 conn(0x7faf78068660 msgr2=0x7faf780a3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:50.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 -- 172.21.15.143:0/2706324732 shutdown_connections 2023-09-29T23:29:50.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.080+0000 7faf7efdf700 1 -- 172.21.15.143:0/2706324732 wait complete. 2023-09-29T23:29:50.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705672 got 38654705672 for osd.3 2023-09-29T23:29:50.094 DEBUG:teuthology.parallel:result is None 2023-09-29T23:29:50.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.1 2023-09-29T23:29:50.099 DEBUG:teuthology.parallel:result is None 2023-09-29T23:29:50.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.100+0000 7efcfe60f700 1 -- 172.21.15.143:0/2724002504 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efcf809ee70 con 0x7efcf809f820 2023-09-29T23:29:50.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcdffff700 1 -- 172.21.15.143:0/2724002504 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efcf008a620 con 0x7efcf809f820 2023-09-29T23:29:50.105 INFO:teuthology.orchestra.run.smithi143.stdout:38654705671 2023-09-29T23:29:50.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 10 mgrc shutdown 2023-09-29T23:29:50.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 -- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efce408e8e0 msgr2=0x7efce4090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efce408e8e0 0x7efce4090d90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efce8003a80 tx=0x7efce8007ec0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 -- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcf809f820 msgr2=0x7efcf8152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcf809f820 0x7efcf8152ff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efcf000b660 tx=0x7efcf001a000 comp rx=0 tx=0).stop 2023-09-29T23:29:50.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 -- 172.21.15.143:0/2724002504 shutdown_connections 2023-09-29T23:29:50.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efce408e8e0 0x7efce4090d90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efcf809f820 0x7efcf8152ff0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcf809e9d0 0x7efcf8143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 --2- 172.21.15.143:0/2724002504 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efcf806f060 0x7efcf806ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 -- 172.21.15.143:0/2724002504 >> 172.21.15.143:0/2724002504 conn(0x7efcf80b2ee0 msgr2=0x7efcf80a0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:50.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 -- 172.21.15.143:0/2724002504 shutdown_connections 2023-09-29T23:29:50.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.104+0000 7efcfe60f700 1 -- 172.21.15.143:0/2724002504 wait complete. 2023-09-29T23:29:50.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705671 got 38654705671 for osd.2 2023-09-29T23:29:50.118 DEBUG:teuthology.parallel:result is None 2023-09-29T23:29:50.118 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-29T23:29:50.118 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T23:29:50.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.212+0000 7f7b51066700 1 Processor -- start 2023-09-29T23:29:50.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.212+0000 7f7b51066700 1 -- start start 2023-09-29T23:29:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7b4c0b1af0 0x7f7b4c0a7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7b4c1415d0 0x7f7b4c148910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b4c0a7920 con 0x7f7b4c134610 2023-09-29T23:29:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4c06bc10 con 0x7f7b4c0b1af0 2023-09-29T23:29:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b4c141e90 con 0x7f7b4c1415d0 2023-09-29T23:29:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7b4c0b1af0 0x7f7b4c0a7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4ad9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f7b4c0b1af0 0x7f7b4c0a7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34688/0 (socket says 172.21.15.143:34688) 2023-09-29T23:29:50.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4ad9d700 1 -- 172.21.15.143:0/876768397 learned_addr learned my addr 172.21.15.143:0/876768397 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4b59e700 1 --2- 172.21.15.143:0/876768397 >> v2:172.21.15.156:3300/0 conn(0x7f7b4c1415d0 0x7f7b4c148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4ad9d700 1 -- 172.21.15.143:0/876768397 >> v2:172.21.15.156:3300/0 conn(0x7f7b4c1415d0 msgr2=0x7f7b4c148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4ad9d700 1 --2- 172.21.15.143:0/876768397 >> v2:172.21.15.156:3300/0 conn(0x7f7b4c1415d0 0x7f7b4c148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4ad9d700 1 -- 172.21.15.143:0/876768397 >> v1:172.21.15.156:6789/0 conn(0x7f7b4c134610 legacy=0x7f7b4c0673a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4ad9d700 1 -- 172.21.15.143:0/876768397 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b4c142150 con 0x7f7b4c0b1af0 2023-09-29T23:29:50.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4ad9d700 1 --2- 172.21.15.143:0/876768397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b4c0b1af0 0x7f7b4c0a7390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0a4a10 tx=0x7f7b3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3a530eb75e1383 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:50.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4a59c700 1 -- 172.21.15.143:0/876768397 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c009e70 con 0x7f7b4c0b1af0 2023-09-29T23:29:50.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4a59c700 1 -- 172.21.15.143:0/876768397 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c00b860 con 0x7f7b4c0b1af0 2023-09-29T23:29:50.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b4a59c700 1 -- 172.21.15.143:0/876768397 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c00bc00 con 0x7f7b4c0b1af0 2023-09-29T23:29:50.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 -- 172.21.15.143:0/876768397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b4c0b1af0 msgr2=0x7f7b4c0a7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 --2- 172.21.15.143:0/876768397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b4c0b1af0 0x7f7b4c0a7390 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0a4a10 tx=0x7f7b3c009250 comp rx=0 tx=0).stop 2023-09-29T23:29:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 -- 172.21.15.143:0/876768397 shutdown_connections 2023-09-29T23:29:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 --2- 172.21.15.143:0/876768397 >> v2:172.21.15.156:3300/0 conn(0x7f7b4c1415d0 0x7f7b4c148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 --2- 172.21.15.143:0/876768397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b4c0b1af0 0x7f7b4c0a7390 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 -- 172.21.15.143:0/876768397 >> 172.21.15.143:0/876768397 conn(0x7f7b4c0b5420 msgr2=0x7f7b4c066c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 -- 172.21.15.143:0/876768397 shutdown_connections 2023-09-29T23:29:50.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.216+0000 7f7b51066700 1 -- 172.21.15.143:0/876768397 wait complete. 2023-09-29T23:29:50.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b51066700 1 Processor -- start 2023-09-29T23:29:50.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b51066700 1 -- start start 2023-09-29T23:29:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b51066700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c0b1af0 0x7f7b4c133890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b51066700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b4c134610 0x7f7b4c12edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b51066700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7b4c1415d0 0x7f7b4c0ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b51066700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c14ef90 con 0x7f7b4c134610 2023-09-29T23:29:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b51066700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c14e260 con 0x7f7b4c0b1af0 2023-09-29T23:29:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b51066700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c14e520 con 0x7f7b4c1415d0 2023-09-29T23:29:50.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b43fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b4c134610 0x7f7b4c12edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b4b59e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7b4c1415d0 0x7f7b4c0ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b4b59e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7b4c1415d0 0x7f7b4c0ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60538/0 (socket says 172.21.15.143:60538) 2023-09-29T23:29:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b43fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b4c134610 0x7f7b4c12edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34696/0 (socket says 172.21.15.143:34696) 2023-09-29T23:29:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b4b59e700 1 -- 172.21.15.143:0/1704993054 learned_addr learned my addr 172.21.15.143:0/1704993054 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.220+0000 7f7b4ad9d700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c0b1af0 0x7f7b4c133890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b4b59e700 1 -- 172.21.15.143:0/1704993054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c0b1af0 msgr2=0x7f7b4c133890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b4b59e700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c0b1af0 0x7f7b4c133890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b4b59e700 1 -- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b4c134610 msgr2=0x7f7b4c12edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b4b59e700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b4c134610 0x7f7b4c12edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b4b59e700 1 -- 172.21.15.143:0/1704993054 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c0094f0 con 0x7f7b4c1415d0 2023-09-29T23:29:50.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b4b59e700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7b4c1415d0 0x7f7b4c0ae5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009e90 tx=0x7f7b4400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:50.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b42ffd700 1 -- 172.21.15.143:0/1704993054 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400cb70 con 0x7f7b4c1415d0 2023-09-29T23:29:50.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b42ffd700 1 -- 172.21.15.143:0/1704993054 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b4400f040 con 0x7f7b4c1415d0 2023-09-29T23:29:50.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b51066700 1 -- 172.21.15.143:0/1704993054 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4c0aac60 con 0x7f7b4c1415d0 2023-09-29T23:29:50.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b42ffd700 1 -- 172.21.15.143:0/1704993054 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b44014480 con 0x7f7b4c1415d0 2023-09-29T23:29:50.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b51066700 1 -- 172.21.15.143:0/1704993054 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c14dda0 con 0x7f7b4c1415d0 2023-09-29T23:29:50.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.224+0000 7f7b51066700 1 -- 172.21.15.143:0/1704993054 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4c14e020 con 0x7f7b4c1415d0 2023-09-29T23:29:50.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.228+0000 7f7b42ffd700 1 -- 172.21.15.143:0/1704993054 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f7b44014830 con 0x7f7b4c1415d0 2023-09-29T23:29:50.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.228+0000 7f7b42ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:50.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.228+0000 7f7b42ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:50.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.228+0000 7f7b42ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:50.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.228+0000 7f7b42ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:50.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.228+0000 7f7b42ffd700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7b2c08e8b0 0x7f7b2c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.228+0000 7f7b4ad9d700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7b2c08e8b0 0x7f7b2c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.228+0000 7f7b42ffd700 1 -- 172.21.15.143:0/1704993054 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f7b440b5150 con 0x7f7b4c1415d0 2023-09-29T23:29:50.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.228+0000 7f7b4ad9d700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7b2c08e8b0 0x7f7b2c090d60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0a4a10 tx=0x7f7b3c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:50.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.236+0000 7f7b42ffd700 1 -- 172.21.15.143:0/1704993054 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7b4c14e020 con 0x7f7b4c1415d0 2023-09-29T23:29:50.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.396+0000 7f7b51066700 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2023-09-29T23:29:50.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.396+0000 7f7b51066700 1 -- 172.21.15.143:0/1704993054 --> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b4c14e020 con 0x7f7b2c08e8b0 2023-09-29T23:29:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.404+0000 7f7b42ffd700 1 -- 172.21.15.143:0/1704993054 <== mgr.4101 v2:172.21.15.143:6808/27602 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+269430 (secure 0 0 0) 0x7f7b4c14e020 con 0x7f7b2c08e8b0 2023-09-29T23:29:50.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.404+0000 7f7b42ffd700 20 mgrc handle_command_reply tid 0 r 0 2023-09-29T23:29:50.410 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:50.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 10 mgrc shutdown 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 -- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7b2c08e8b0 msgr2=0x7f7b2c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7b2c08e8b0 0x7f7b2c090d60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c0a4a10 tx=0x7f7b3c017000 comp rx=0 tx=0).stop 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 -- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7b4c1415d0 msgr2=0x7f7b4c0ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7b4c1415d0 0x7f7b4c0ae5a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7b44009e90 tx=0x7f7b4400b040 comp rx=0 tx=0).stop 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 -- 172.21.15.143:0/1704993054 shutdown_connections 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7b2c08e8b0 0x7f7b2c090d60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7b4c1415d0 0x7f7b4c0ae5a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7b4c134610 0x7f7b4c12edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 --2- 172.21.15.143:0/1704993054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b4c0b1af0 0x7f7b4c133890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 -- 172.21.15.143:0/1704993054 >> 172.21.15.143:0/1704993054 conn(0x7f7b4c0b5420 msgr2=0x7f7b4c0a4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 -- 172.21.15.143:0/1704993054 shutdown_connections 2023-09-29T23:29:50.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.408+0000 7f7b51066700 1 -- 172.21.15.143:0/1704993054 wait complete. 2023-09-29T23:29:50.414 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T23:29:50.423 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-09-29T23:29:48.464782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":47376,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":377439984,"statfs":{"total":386547056640,"available":386498543616,"internally_reserved":0,"allocated":901120,"data_stored":1426840,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.871702"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.639872+0000","last_change":"2023-09-29T23:29:39.634761+0000","last_active":"2023-09-29T23:29:39.639872+0000","last_peered":"2023-09-29T23:29:39.639872+0000","last_clean":"2023-09-29T23:29:39.639872+0000","last_became_active":"2023-09-29T23:29:39.634501+0000","last_became_peered":"2023-09-29T23:29:39.634501+0000","last_unstale":"2023-09-29T23:29:39.639872+0000","last_undegraded":"2023-09-29T23:29:39.639872+0000","last_fullsized":"2023-09-29T23:29:39.639872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:20:37.281771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620836+0000","last_change":"2023-09-29T23:29:39.615851+0000","last_active":"2023-09-29T23:29:39.620836+0000","last_peered":"2023-09-29T23:29:39.620836+0000","last_clean":"2023-09-29T23:29:39.620836+0000","last_became_active":"2023-09-29T23:29:39.615521+0000","last_became_peered":"2023-09-29T23:29:39.615521+0000","last_unstale":"2023-09-29T23:29:39.620836+0000","last_undegraded":"2023-09-29T23:29:39.620836+0000","last_fullsized":"2023-09-29T23:29:39.620836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:17:42.897947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647121+0000","last_change":"2023-09-29T23:29:39.636680+0000","last_active":"2023-09-29T23:29:39.647121+0000","last_peered":"2023-09-29T23:29:39.647121+0000","last_clean":"2023-09-29T23:29:39.647121+0000","last_became_active":"2023-09-29T23:29:39.634721+0000","last_became_peered":"2023-09-29T23:29:39.634721+0000","last_unstale":"2023-09-29T23:29:39.647121+0000","last_undegraded":"2023-09-29T23:29:39.647121+0000","last_fullsized":"2023-09-29T23:29:39.647121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:36.503704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625198+0000","last_change":"2023-09-29T23:29:39.612565+0000","last_active":"2023-09-29T23:29:39.625198+0000","last_peered":"2023-09-29T23:29:39.625198+0000","last_clean":"2023-09-29T23:29:39.625198+0000","last_became_active":"2023-09-29T23:29:39.612183+0000","last_became_peered":"2023-09-29T23:29:39.612183+0000","last_unstale":"2023-09-29T23:29:39.625198+0000","last_undegraded":"2023-09-29T23:29:39.625198+0000","last_fullsized":"2023-09-29T23:29:39.625198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:42.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645186+0000","last_change":"2023-09-29T23:29:39.633553+0000","last_active":"2023-09-29T23:29:39.645186+0000","last_peered":"2023-09-29T23:29:39.645186+0000","last_clean":"2023-09-29T23:29:39.645186+0000","last_became_active":"2023-09-29T23:29:39.633127+0000","last_became_peered":"2023-09-29T23:29:39.633127+0000","last_unstale":"2023-09-29T23:29:39.645186+0000","last_undegraded":"2023-09-29T23:29:39.645186+0000","last_fullsized":"2023-09-29T23:29:39.645186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:47:58.354132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645627+0000","last_change":"2023-09-29T23:29:39.636332+0000","last_active":"2023-09-29T23:29:39.645627+0000","last_peered":"2023-09-29T23:29:39.645627+0000","last_clean":"2023-09-29T23:29:39.645627+0000","last_became_active":"2023-09-29T23:29:39.636025+0000","last_became_peered":"2023-09-29T23:29:39.636025+0000","last_unstale":"2023-09-29T23:29:39.645627+0000","last_undegraded":"2023-09-29T23:29:39.645627+0000","last_fullsized":"2023-09-29T23:29:39.645627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:33.457443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645224+0000","last_change":"2023-09-29T23:29:39.634398+0000","last_active":"2023-09-29T23:29:39.645224+0000","last_peered":"2023-09-29T23:29:39.645224+0000","last_clean":"2023-09-29T23:29:39.645224+0000","last_became_active":"2023-09-29T23:29:39.633340+0000","last_became_peered":"2023-09-29T23:29:39.633340+0000","last_unstale":"2023-09-29T23:29:39.645224+0000","last_undegraded":"2023-09-29T23:29:39.645224+0000","last_fullsized":"2023-09-29T23:29:39.645224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:33:09.568111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629860+0000","last_change":"2023-09-29T23:29:39.618059+0000","last_active":"2023-09-29T23:29:39.629860+0000","last_peered":"2023-09-29T23:29:39.629860+0000","last_clean":"2023-09-29T23:29:39.629860+0000","last_became_active":"2023-09-29T23:29:39.617767+0000","last_became_peered":"2023-09-29T23:29:39.617767+0000","last_unstale":"2023-09-29T23:29:39.629860+0000","last_undegraded":"2023-09-29T23:29:39.629860+0000","last_fullsized":"2023-09-29T23:29:39.629860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:55:52.762563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.630737+0000","last_change":"2023-09-29T23:29:39.626861+0000","last_active":"2023-09-29T23:29:39.630737+0000","last_peered":"2023-09-29T23:29:39.630737+0000","last_clean":"2023-09-29T23:29:39.630737+0000","last_became_active":"2023-09-29T23:29:39.626482+0000","last_became_peered":"2023-09-29T23:29:39.626482+0000","last_unstale":"2023-09-29T23:29:39.630737+0000","last_undegraded":"2023-09-29T23:29:39.630737+0000","last_fullsized":"2023-09-29T23:29:39.630737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:48.157072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641519+0000","last_change":"2023-09-29T23:29:39.640158+0000","last_active":"2023-09-29T23:29:39.641519+0000","last_peered":"2023-09-29T23:29:39.641519+0000","last_clean":"2023-09-29T23:29:39.641519+0000","last_became_active":"2023-09-29T23:29:39.639864+0000","last_became_peered":"2023-09-29T23:29:39.639864+0000","last_unstale":"2023-09-29T23:29:39.641519+0000","last_undegraded":"2023-09-29T23:29:39.641519+0000","last_fullsized":"2023-09-29T23:29:39.641519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:46:27.931849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625089+0000","last_change":"2023-09-29T23:29:39.612352+0000","last_active":"2023-09-29T23:29:39.625089+0000","last_peered":"2023-09-29T23:29:39.625089+0000","last_clean":"2023-09-29T23:29:39.625089+0000","last_became_active":"2023-09-29T23:29:39.611961+0000","last_became_peered":"2023-09-29T23:29:39.611961+0000","last_unstale":"2023-09-29T23:29:39.625089+0000","last_undegraded":"2023-09-29T23:29:39.625089+0000","last_fullsized":"2023-09-29T23:29:39.625089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:46:32.838187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638115+0000","last_change":"2023-09-29T23:29:39.632780+0000","last_active":"2023-09-29T23:29:39.638115+0000","last_peered":"2023-09-29T23:29:39.638115+0000","last_clean":"2023-09-29T23:29:39.638115+0000","last_became_active":"2023-09-29T23:29:39.632492+0000","last_became_peered":"2023-09-29T23:29:39.632492+0000","last_unstale":"2023-09-29T23:29:39.638115+0000","last_undegraded":"2023-09-29T23:29:39.638115+0000","last_fullsized":"2023-09-29T23:29:39.638115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:14:21.471042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.636401+0000","last_change":"2023-09-29T23:29:39.633421+0000","last_active":"2023-09-29T23:29:39.636401+0000","last_peered":"2023-09-29T23:29:39.636401+0000","last_clean":"2023-09-29T23:29:39.636401+0000","last_became_active":"2023-09-29T23:29:39.633127+0000","last_became_peered":"2023-09-29T23:29:39.633127+0000","last_unstale":"2023-09-29T23:29:39.636401+0000","last_undegraded":"2023-09-29T23:29:39.636401+0000","last_fullsized":"2023-09-29T23:29:39.636401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:58:35.776573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629920+0000","last_change":"2023-09-29T23:29:39.621931+0000","last_active":"2023-09-29T23:29:39.629920+0000","last_peered":"2023-09-29T23:29:39.629920+0000","last_clean":"2023-09-29T23:29:39.629920+0000","last_became_active":"2023-09-29T23:29:39.621616+0000","last_became_peered":"2023-09-29T23:29:39.621616+0000","last_unstale":"2023-09-29T23:29:39.629920+0000","last_undegraded":"2023-09-29T23:29:39.629920+0000","last_fullsized":"2023-09-29T23:29:39.629920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:53:43.483894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629993+0000","last_change":"2023-09-29T23:29:39.622200+0000","last_active":"2023-09-29T23:29:39.629993+0000","last_peered":"2023-09-29T23:29:39.629993+0000","last_clean":"2023-09-29T23:29:39.629993+0000","last_became_active":"2023-09-29T23:29:39.621385+0000","last_became_peered":"2023-09-29T23:29:39.621385+0000","last_unstale":"2023-09-29T23:29:39.629993+0000","last_undegraded":"2023-09-29T23:29:39.629993+0000","last_fullsized":"2023-09-29T23:29:39.629993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:17:19.555819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645817+0000","last_change":"2023-09-29T23:29:39.638607+0000","last_active":"2023-09-29T23:29:39.645817+0000","last_peered":"2023-09-29T23:29:39.645817+0000","last_clean":"2023-09-29T23:29:39.645817+0000","last_became_active":"2023-09-29T23:29:39.635921+0000","last_became_peered":"2023-09-29T23:29:39.635921+0000","last_unstale":"2023-09-29T23:29:39.645817+0000","last_undegraded":"2023-09-29T23:29:39.645817+0000","last_fullsized":"2023-09-29T23:29:39.645817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:27:52.497036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.631406+0000","last_change":"2023-09-29T23:29:39.622234+0000","last_active":"2023-09-29T23:29:39.631406+0000","last_peered":"2023-09-29T23:29:39.631406+0000","last_clean":"2023-09-29T23:29:39.631406+0000","last_became_active":"2023-09-29T23:29:39.621861+0000","last_became_peered":"2023-09-29T23:29:39.621861+0000","last_unstale":"2023-09-29T23:29:39.631406+0000","last_undegraded":"2023-09-29T23:29:39.631406+0000","last_fullsized":"2023-09-29T23:29:39.631406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:54:29.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.646545+0000","last_change":"2023-09-29T23:29:39.639501+0000","last_active":"2023-09-29T23:29:39.646545+0000","last_peered":"2023-09-29T23:29:39.646545+0000","last_clean":"2023-09-29T23:29:39.646545+0000","last_became_active":"2023-09-29T23:29:39.638774+0000","last_became_peered":"2023-09-29T23:29:39.638774+0000","last_unstale":"2023-09-29T23:29:39.646545+0000","last_undegraded":"2023-09-29T23:29:39.646545+0000","last_fullsized":"2023-09-29T23:29:39.646545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:37:30.614407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.636591+0000","last_change":"2023-09-29T23:29:39.633491+0000","last_active":"2023-09-29T23:29:39.636591+0000","last_peered":"2023-09-29T23:29:39.636591+0000","last_clean":"2023-09-29T23:29:39.636591+0000","last_became_active":"2023-09-29T23:29:39.633174+0000","last_became_peered":"2023-09-29T23:29:39.633174+0000","last_unstale":"2023-09-29T23:29:39.636591+0000","last_undegraded":"2023-09-29T23:29:39.636591+0000","last_fullsized":"2023-09-29T23:29:39.636591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:36:55.020338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625219+0000","last_change":"2023-09-29T23:29:39.613672+0000","last_active":"2023-09-29T23:29:39.625219+0000","last_peered":"2023-09-29T23:29:39.625219+0000","last_clean":"2023-09-29T23:29:39.625219+0000","last_became_active":"2023-09-29T23:29:39.612667+0000","last_became_peered":"2023-09-29T23:29:39.612667+0000","last_unstale":"2023-09-29T23:29:39.625219+0000","last_undegraded":"2023-09-29T23:29:39.625219+0000","last_fullsized":"2023-09-29T23:29:39.625219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:47:46.021660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641070+0000","last_change":"2023-09-29T23:29:39.639206+0000","last_active":"2023-09-29T23:29:39.641070+0000","last_peered":"2023-09-29T23:29:39.641070+0000","last_clean":"2023-09-29T23:29:39.641070+0000","last_became_active":"2023-09-29T23:29:39.638930+0000","last_became_peered":"2023-09-29T23:29:39.638930+0000","last_unstale":"2023-09-29T23:29:39.641070+0000","last_undegraded":"2023-09-29T23:29:39.641070+0000","last_fullsized":"2023-09-29T23:29:39.641070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:34:56.298163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629896+0000","last_change":"2023-09-29T23:29:39.621733+0000","last_active":"2023-09-29T23:29:39.629896+0000","last_peered":"2023-09-29T23:29:39.629896+0000","last_clean":"2023-09-29T23:29:39.629896+0000","last_became_active":"2023-09-29T23:29:39.621410+0000","last_became_peered":"2023-09-29T23:29:39.621410+0000","last_unstale":"2023-09-29T23:29:39.629896+0000","last_undegraded":"2023-09-29T23:29:39.629896+0000","last_fullsized":"2023-09-29T23:29:39.629896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:13:11.017795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645339+0000","last_change":"2023-09-29T23:29:39.635888+0000","last_active":"2023-09-29T23:29:39.645339+0000","last_peered":"2023-09-29T23:29:39.645339+0000","last_clean":"2023-09-29T23:29:39.645339+0000","last_became_active":"2023-09-29T23:29:39.634120+0000","last_became_peered":"2023-09-29T23:29:39.634120+0000","last_unstale":"2023-09-29T23:29:39.645339+0000","last_undegraded":"2023-09-29T23:29:39.645339+0000","last_fullsized":"2023-09-29T23:29:39.645339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:16:01.339180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625208+0000","last_change":"2023-09-29T23:29:39.612568+0000","last_active":"2023-09-29T23:29:39.625208+0000","last_peered":"2023-09-29T23:29:39.625208+0000","last_clean":"2023-09-29T23:29:39.625208+0000","last_became_active":"2023-09-29T23:29:39.612349+0000","last_became_peered":"2023-09-29T23:29:39.612349+0000","last_unstale":"2023-09-29T23:29:39.625208+0000","last_undegraded":"2023-09-29T23:29:39.625208+0000","last_fullsized":"2023-09-29T23:29:39.625208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:36:55.020338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641527+0000","last_change":"2023-09-29T23:29:39.640244+0000","last_active":"2023-09-29T23:29:39.641527+0000","last_peered":"2023-09-29T23:29:39.641527+0000","last_clean":"2023-09-29T23:29:39.641527+0000","last_became_active":"2023-09-29T23:29:39.639928+0000","last_became_peered":"2023-09-29T23:29:39.639928+0000","last_unstale":"2023-09-29T23:29:39.641527+0000","last_undegraded":"2023-09-29T23:29:39.641527+0000","last_fullsized":"2023-09-29T23:29:39.641527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:20:35.058663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.610083+0000","last_change":"2023-09-29T23:29:39.609392+0000","last_active":"2023-09-29T23:29:39.610083+0000","last_peered":"2023-09-29T23:29:39.610083+0000","last_clean":"2023-09-29T23:29:39.610083+0000","last_became_active":"2023-09-29T23:29:39.609137+0000","last_became_peered":"2023-09-29T23:29:39.609137+0000","last_unstale":"2023-09-29T23:29:39.610083+0000","last_undegraded":"2023-09-29T23:29:39.610083+0000","last_fullsized":"2023-09-29T23:29:39.610083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:14:21.471042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647538+0000","last_change":"2023-09-29T23:29:39.646351+0000","last_active":"2023-09-29T23:29:39.647538+0000","last_peered":"2023-09-29T23:29:39.647538+0000","last_clean":"2023-09-29T23:29:39.647538+0000","last_became_active":"2023-09-29T23:29:39.646091+0000","last_became_peered":"2023-09-29T23:29:39.646091+0000","last_unstale":"2023-09-29T23:29:39.647538+0000","last_undegraded":"2023-09-29T23:29:39.647538+0000","last_fullsized":"2023-09-29T23:29:39.647538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:41:52.519640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.606960+0000","last_change":"2023-09-29T23:29:39.606266+0000","last_active":"2023-09-29T23:29:39.606960+0000","last_peered":"2023-09-29T23:29:39.606960+0000","last_clean":"2023-09-29T23:29:39.606960+0000","last_became_active":"2023-09-29T23:29:39.606025+0000","last_became_peered":"2023-09-29T23:29:39.606025+0000","last_unstale":"2023-09-29T23:29:39.606960+0000","last_undegraded":"2023-09-29T23:29:39.606960+0000","last_fullsized":"2023-09-29T23:29:39.606960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:17:42.897947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637537+0000","last_change":"2023-09-29T23:29:39.634587+0000","last_active":"2023-09-29T23:29:39.637537+0000","last_peered":"2023-09-29T23:29:39.637537+0000","last_clean":"2023-09-29T23:29:39.637537+0000","last_became_active":"2023-09-29T23:29:39.634344+0000","last_became_peered":"2023-09-29T23:29:39.634344+0000","last_unstale":"2023-09-29T23:29:39.637537+0000","last_undegraded":"2023-09-29T23:29:39.637537+0000","last_fullsized":"2023-09-29T23:29:39.637537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:20:37.281771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637839+0000","last_change":"2023-09-29T23:29:39.632991+0000","last_active":"2023-09-29T23:29:39.637839+0000","last_peered":"2023-09-29T23:29:39.637839+0000","last_clean":"2023-09-29T23:29:39.637839+0000","last_became_active":"2023-09-29T23:29:39.632721+0000","last_became_peered":"2023-09-29T23:29:39.632721+0000","last_unstale":"2023-09-29T23:29:39.637839+0000","last_undegraded":"2023-09-29T23:29:39.637839+0000","last_fullsized":"2023-09-29T23:29:39.637839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:25:55.111560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.618030+0000","last_change":"2023-09-29T23:29:39.616390+0000","last_active":"2023-09-29T23:29:39.618030+0000","last_peered":"2023-09-29T23:29:39.618030+0000","last_clean":"2023-09-29T23:29:39.618030+0000","last_became_active":"2023-09-29T23:29:39.615974+0000","last_became_peered":"2023-09-29T23:29:39.615974+0000","last_unstale":"2023-09-29T23:29:39.618030+0000","last_undegraded":"2023-09-29T23:29:39.618030+0000","last_fullsized":"2023-09-29T23:29:39.618030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:05:52.183625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.630420+0000","last_change":"2023-09-29T23:29:39.625067+0000","last_active":"2023-09-29T23:29:39.630420+0000","last_peered":"2023-09-29T23:29:39.630420+0000","last_clean":"2023-09-29T23:29:39.630420+0000","last_became_active":"2023-09-29T23:29:39.624793+0000","last_became_peered":"2023-09-29T23:29:39.624793+0000","last_unstale":"2023-09-29T23:29:39.630420+0000","last_undegraded":"2023-09-29T23:29:39.630420+0000","last_fullsized":"2023-09-29T23:29:39.630420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:48.157072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645155+0000","last_change":"2023-09-29T23:29:39.633681+0000","last_active":"2023-09-29T23:29:39.645155+0000","last_peered":"2023-09-29T23:29:39.645155+0000","last_clean":"2023-09-29T23:29:39.645155+0000","last_became_active":"2023-09-29T23:29:39.633400+0000","last_became_peered":"2023-09-29T23:29:39.633400+0000","last_unstale":"2023-09-29T23:29:39.645155+0000","last_undegraded":"2023-09-29T23:29:39.645155+0000","last_fullsized":"2023-09-29T23:29:39.645155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:29:56.190518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638194+0000","last_change":"2023-09-29T23:29:39.272166+0000","last_active":"2023-09-29T23:29:39.638194+0000","last_peered":"2023-09-29T23:29:39.638194+0000","last_clean":"2023-09-29T23:29:39.638194+0000","last_became_active":"2023-09-29T23:29:39.271616+0000","last_became_peered":"2023-09-29T23:29:39.271616+0000","last_unstale":"2023-09-29T23:29:39.638194+0000","last_undegraded":"2023-09-29T23:29:39.638194+0000","last_fullsized":"2023-09-29T23:29:39.638194+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:14:20.467285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647140+0000","last_change":"2023-09-29T23:29:39.639338+0000","last_active":"2023-09-29T23:29:39.647140+0000","last_peered":"2023-09-29T23:29:39.647140+0000","last_clean":"2023-09-29T23:29:39.647140+0000","last_became_active":"2023-09-29T23:29:39.639030+0000","last_became_peered":"2023-09-29T23:29:39.639030+0000","last_unstale":"2023-09-29T23:29:39.647140+0000","last_undegraded":"2023-09-29T23:29:39.647140+0000","last_fullsized":"2023-09-29T23:29:39.647140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:46:49.801617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.598222+0000","last_change":"2023-09-29T23:29:39.269808+0000","last_active":"2023-09-29T23:29:39.598222+0000","last_peered":"2023-09-29T23:29:39.598222+0000","last_clean":"2023-09-29T23:29:39.598222+0000","last_became_active":"2023-09-29T23:29:39.269509+0000","last_became_peered":"2023-09-29T23:29:39.269509+0000","last_unstale":"2023-09-29T23:29:39.598222+0000","last_undegraded":"2023-09-29T23:29:39.598222+0000","last_fullsized":"2023-09-29T23:29:39.598222+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:47:31.314319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.646474+0000","last_change":"2023-09-29T23:29:39.639099+0000","last_active":"2023-09-29T23:29:39.646474+0000","last_peered":"2023-09-29T23:29:39.646474+0000","last_clean":"2023-09-29T23:29:39.646474+0000","last_became_active":"2023-09-29T23:29:39.636608+0000","last_became_peered":"2023-09-29T23:29:39.636608+0000","last_unstale":"2023-09-29T23:29:39.646474+0000","last_undegraded":"2023-09-29T23:29:39.646474+0000","last_fullsized":"2023-09-29T23:29:39.646474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:24:20.819335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620024+0000","last_change":"2023-09-29T23:29:39.273761+0000","last_active":"2023-09-29T23:29:39.620024+0000","last_peered":"2023-09-29T23:29:39.620024+0000","last_clean":"2023-09-29T23:29:39.620024+0000","last_became_active":"2023-09-29T23:29:39.273376+0000","last_became_peered":"2023-09-29T23:29:39.273376+0000","last_unstale":"2023-09-29T23:29:39.620024+0000","last_undegraded":"2023-09-29T23:29:39.620024+0000","last_fullsized":"2023-09-29T23:29:39.620024+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:59:21.890466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.636454+0000","last_change":"2023-09-29T23:29:39.631948+0000","last_active":"2023-09-29T23:29:39.636454+0000","last_peered":"2023-09-29T23:29:39.636454+0000","last_clean":"2023-09-29T23:29:39.636454+0000","last_became_active":"2023-09-29T23:29:39.631531+0000","last_became_peered":"2023-09-29T23:29:39.631531+0000","last_unstale":"2023-09-29T23:29:39.636454+0000","last_undegraded":"2023-09-29T23:29:39.636454+0000","last_fullsized":"2023-09-29T23:29:39.636454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:04:09.469416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645828+0000","last_change":"2023-09-29T23:29:39.259965+0000","last_active":"2023-09-29T23:29:39.645828+0000","last_peered":"2023-09-29T23:29:39.645828+0000","last_clean":"2023-09-29T23:29:39.645828+0000","last_became_active":"2023-09-29T23:29:39.259617+0000","last_became_peered":"2023-09-29T23:29:39.259617+0000","last_unstale":"2023-09-29T23:29:39.645828+0000","last_undegraded":"2023-09-29T23:29:39.645828+0000","last_fullsized":"2023-09-29T23:29:39.645828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:43:32.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.630444+0000","last_change":"2023-09-29T23:29:39.262801+0000","last_active":"2023-09-29T23:29:39.630444+0000","last_peered":"2023-09-29T23:29:39.630444+0000","last_clean":"2023-09-29T23:29:39.630444+0000","last_became_active":"2023-09-29T23:29:39.259692+0000","last_became_peered":"2023-09-29T23:29:39.259692+0000","last_unstale":"2023-09-29T23:29:39.630444+0000","last_undegraded":"2023-09-29T23:29:39.630444+0000","last_fullsized":"2023-09-29T23:29:39.630444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:43:32.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645777+0000","last_change":"2023-09-29T23:29:39.636515+0000","last_active":"2023-09-29T23:29:39.645777+0000","last_peered":"2023-09-29T23:29:39.645777+0000","last_clean":"2023-09-29T23:29:39.645777+0000","last_became_active":"2023-09-29T23:29:39.633532+0000","last_became_peered":"2023-09-29T23:29:39.633532+0000","last_unstale":"2023-09-29T23:29:39.645777+0000","last_undegraded":"2023-09-29T23:29:39.645777+0000","last_fullsized":"2023-09-29T23:29:39.645777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:53:43.483894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625041+0000","last_change":"2023-09-29T23:29:39.268225+0000","last_active":"2023-09-29T23:29:39.625041+0000","last_peered":"2023-09-29T23:29:39.625041+0000","last_clean":"2023-09-29T23:29:39.625041+0000","last_became_active":"2023-09-29T23:29:39.267910+0000","last_became_peered":"2023-09-29T23:29:39.267910+0000","last_unstale":"2023-09-29T23:29:39.625041+0000","last_undegraded":"2023-09-29T23:29:39.625041+0000","last_fullsized":"2023-09-29T23:29:39.625041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:58:28.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.624786+0000","last_change":"2023-09-29T23:29:39.612447+0000","last_active":"2023-09-29T23:29:39.624786+0000","last_peered":"2023-09-29T23:29:39.624786+0000","last_clean":"2023-09-29T23:29:39.624786+0000","last_became_active":"2023-09-29T23:29:39.611905+0000","last_became_peered":"2023-09-29T23:29:39.611905+0000","last_unstale":"2023-09-29T23:29:39.624786+0000","last_undegraded":"2023-09-29T23:29:39.624786+0000","last_fullsized":"2023-09-29T23:29:39.624786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:31.379677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645233+0000","last_change":"2023-09-29T23:29:39.270818+0000","last_active":"2023-09-29T23:29:39.645233+0000","last_peered":"2023-09-29T23:29:39.645233+0000","last_clean":"2023-09-29T23:29:39.645233+0000","last_became_active":"2023-09-29T23:29:39.270419+0000","last_became_peered":"2023-09-29T23:29:39.270419+0000","last_unstale":"2023-09-29T23:29:39.645233+0000","last_undegraded":"2023-09-29T23:29:39.645233+0000","last_fullsized":"2023-09-29T23:29:39.645233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:39:03.054065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638105+0000","last_change":"2023-09-29T23:29:39.634163+0000","last_active":"2023-09-29T23:29:39.638105+0000","last_peered":"2023-09-29T23:29:39.638105+0000","last_clean":"2023-09-29T23:29:39.638105+0000","last_became_active":"2023-09-29T23:29:39.633902+0000","last_became_peered":"2023-09-29T23:29:39.633902+0000","last_unstale":"2023-09-29T23:29:39.638105+0000","last_undegraded":"2023-09-29T23:29:39.638105+0000","last_fullsized":"2023-09-29T23:29:39.638105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:48.157072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.616087+0000","last_change":"2023-09-29T23:29:39.267422+0000","last_active":"2023-09-29T23:29:39.616087+0000","last_peered":"2023-09-29T23:29:39.616087+0000","last_clean":"2023-09-29T23:29:39.616087+0000","last_became_active":"2023-09-29T23:29:39.267119+0000","last_became_peered":"2023-09-29T23:29:39.267119+0000","last_unstale":"2023-09-29T23:29:39.616087+0000","last_undegraded":"2023-09-29T23:29:39.616087+0000","last_fullsized":"2023-09-29T23:29:39.616087+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:29:06.658217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609399+0000","last_change":"2023-09-29T23:29:39.607523+0000","last_active":"2023-09-29T23:29:39.609399+0000","last_peered":"2023-09-29T23:29:39.609399+0000","last_clean":"2023-09-29T23:29:39.609399+0000","last_became_active":"2023-09-29T23:29:39.607172+0000","last_became_peered":"2023-09-29T23:29:39.607172+0000","last_unstale":"2023-09-29T23:29:39.609399+0000","last_undegraded":"2023-09-29T23:29:39.609399+0000","last_fullsized":"2023-09-29T23:29:39.609399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:05:52.183625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.614202+0000","last_change":"2023-09-29T23:29:39.259718+0000","last_active":"2023-09-29T23:29:39.614202+0000","last_peered":"2023-09-29T23:29:39.614202+0000","last_clean":"2023-09-29T23:29:39.614202+0000","last_became_active":"2023-09-29T23:29:39.259429+0000","last_became_peered":"2023-09-29T23:29:39.259429+0000","last_unstale":"2023-09-29T23:29:39.614202+0000","last_undegraded":"2023-09-29T23:29:39.614202+0000","last_fullsized":"2023-09-29T23:29:39.614202+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:18:38.314574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.635728+0000","last_change":"2023-09-29T23:29:39.631644+0000","last_active":"2023-09-29T23:29:39.635728+0000","last_peered":"2023-09-29T23:29:39.635728+0000","last_clean":"2023-09-29T23:29:39.635728+0000","last_became_active":"2023-09-29T23:29:39.630542+0000","last_became_peered":"2023-09-29T23:29:39.630542+0000","last_unstale":"2023-09-29T23:29:39.635728+0000","last_undegraded":"2023-09-29T23:29:39.635728+0000","last_fullsized":"2023-09-29T23:29:39.635728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:40:30.851104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.613760+0000","last_change":"2023-09-29T23:29:39.268889+0000","last_active":"2023-09-29T23:29:39.613760+0000","last_peered":"2023-09-29T23:29:39.613760+0000","last_clean":"2023-09-29T23:29:39.613760+0000","last_became_active":"2023-09-29T23:29:39.268443+0000","last_became_peered":"2023-09-29T23:29:39.268443+0000","last_unstale":"2023-09-29T23:29:39.613760+0000","last_undegraded":"2023-09-29T23:29:39.613760+0000","last_fullsized":"2023-09-29T23:29:39.613760+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:58:34.772816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647157+0000","last_change":"2023-09-29T23:29:39.639561+0000","last_active":"2023-09-29T23:29:39.647157+0000","last_peered":"2023-09-29T23:29:39.647157+0000","last_clean":"2023-09-29T23:29:39.647157+0000","last_became_active":"2023-09-29T23:29:39.639082+0000","last_became_peered":"2023-09-29T23:29:39.639082+0000","last_unstale":"2023-09-29T23:29:39.647157+0000","last_undegraded":"2023-09-29T23:29:39.647157+0000","last_fullsized":"2023-09-29T23:29:39.647157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:15:31.512106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620049+0000","last_change":"2023-09-29T23:29:39.274677+0000","last_active":"2023-09-29T23:29:39.620049+0000","last_peered":"2023-09-29T23:29:39.620049+0000","last_clean":"2023-09-29T23:29:39.620049+0000","last_became_active":"2023-09-29T23:29:39.274395+0000","last_became_peered":"2023-09-29T23:29:39.274395+0000","last_unstale":"2023-09-29T23:29:39.620049+0000","last_undegraded":"2023-09-29T23:29:39.620049+0000","last_fullsized":"2023-09-29T23:29:39.620049+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:48.655823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638074+0000","last_change":"2023-09-29T23:29:39.633620+0000","last_active":"2023-09-29T23:29:39.638074+0000","last_peered":"2023-09-29T23:29:39.638074+0000","last_clean":"2023-09-29T23:29:39.638074+0000","last_became_active":"2023-09-29T23:29:39.633346+0000","last_became_peered":"2023-09-29T23:29:39.633346+0000","last_unstale":"2023-09-29T23:29:39.638074+0000","last_undegraded":"2023-09-29T23:29:39.638074+0000","last_fullsized":"2023-09-29T23:29:39.638074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:59:22.894223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617510+0000","last_change":"2023-09-29T23:29:39.269804+0000","last_active":"2023-09-29T23:29:39.617510+0000","last_peered":"2023-09-29T23:29:39.617510+0000","last_clean":"2023-09-29T23:29:39.617510+0000","last_became_active":"2023-09-29T23:29:39.269530+0000","last_became_peered":"2023-09-29T23:29:39.269530+0000","last_unstale":"2023-09-29T23:29:39.617510+0000","last_undegraded":"2023-09-29T23:29:39.617510+0000","last_fullsized":"2023-09-29T23:29:39.617510+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:47:31.314319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647052+0000","last_change":"2023-09-29T23:29:39.645616+0000","last_active":"2023-09-29T23:29:39.647052+0000","last_peered":"2023-09-29T23:29:39.647052+0000","last_clean":"2023-09-29T23:29:39.647052+0000","last_became_active":"2023-09-29T23:29:39.644961+0000","last_became_peered":"2023-09-29T23:29:39.644961+0000","last_unstale":"2023-09-29T23:29:39.647052+0000","last_undegraded":"2023-09-29T23:29:39.647052+0000","last_fullsized":"2023-09-29T23:29:39.647052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:56:40.493860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":166,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.612078+0000","last_change":"2023-09-29T23:29:29.623917+0000","last_active":"2023-09-29T23:29:39.612078+0000","last_peered":"2023-09-29T23:29:39.612078+0000","last_clean":"2023-09-29T23:29:39.612078+0000","last_became_active":"2023-09-29T23:29:29.621470+0000","last_became_peered":"2023-09-29T23:29:29.621470+0000","last_unstale":"2023-09-29T23:29:39.612078+0000","last_undegraded":"2023-09-29T23:29:39.612078+0000","last_fullsized":"2023-09-29T23:29:39.612078+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_clean_scrub_stamp":"2023-09-29T23:29:28.522101+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:37:09.181891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620013+0000","last_change":"2023-09-29T23:29:35.099334+0000","last_active":"2023-09-29T23:29:39.620013+0000","last_peered":"2023-09-29T23:29:39.620013+0000","last_clean":"2023-09-29T23:29:39.620013+0000","last_became_active":"2023-09-29T23:29:35.098819+0000","last_became_peered":"2023-09-29T23:29:35.098819+0000","last_unstale":"2023-09-29T23:29:39.620013+0000","last_undegraded":"2023-09-29T23:29:39.620013+0000","last_fullsized":"2023-09-29T23:29:39.620013+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:29:21.699082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617326+0000","last_change":"2023-09-29T23:29:39.272150+0000","last_active":"2023-09-29T23:29:39.617326+0000","last_peered":"2023-09-29T23:29:39.617326+0000","last_clean":"2023-09-29T23:29:39.617326+0000","last_became_active":"2023-09-29T23:29:39.271819+0000","last_became_peered":"2023-09-29T23:29:39.271819+0000","last_unstale":"2023-09-29T23:29:39.617326+0000","last_undegraded":"2023-09-29T23:29:39.617326+0000","last_fullsized":"2023-09-29T23:29:39.617326+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:47.153315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647893+0000","last_change":"2023-09-29T23:29:39.646963+0000","last_active":"2023-09-29T23:29:39.647893+0000","last_peered":"2023-09-29T23:29:39.647893+0000","last_clean":"2023-09-29T23:29:39.647893+0000","last_became_active":"2023-09-29T23:29:39.646666+0000","last_became_peered":"2023-09-29T23:29:39.646666+0000","last_unstale":"2023-09-29T23:29:39.647893+0000","last_undegraded":"2023-09-29T23:29:39.647893+0000","last_fullsized":"2023-09-29T23:29:39.647893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:57:10.275706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617601+0000","last_change":"2023-09-29T23:29:35.098951+0000","last_active":"2023-09-29T23:29:39.617601+0000","last_peered":"2023-09-29T23:29:39.617601+0000","last_clean":"2023-09-29T23:29:39.617601+0000","last_became_active":"2023-09-29T23:29:35.098555+0000","last_became_peered":"2023-09-29T23:29:35.098555+0000","last_unstale":"2023-09-29T23:29:39.617601+0000","last_undegraded":"2023-09-29T23:29:39.617601+0000","last_fullsized":"2023-09-29T23:29:39.617601+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:43:14.246995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597396+0000","last_change":"2023-09-29T23:29:39.260929+0000","last_active":"2023-09-29T23:29:39.597396+0000","last_peered":"2023-09-29T23:29:39.597396+0000","last_clean":"2023-09-29T23:29:39.597396+0000","last_became_active":"2023-09-29T23:29:39.260563+0000","last_became_peered":"2023-09-29T23:29:39.260563+0000","last_unstale":"2023-09-29T23:29:39.597396+0000","last_undegraded":"2023-09-29T23:29:39.597396+0000","last_fullsized":"2023-09-29T23:29:39.597396+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:09:46.708805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.646512+0000","last_change":"2023-09-29T23:29:39.639295+0000","last_active":"2023-09-29T23:29:39.646512+0000","last_peered":"2023-09-29T23:29:39.646512+0000","last_clean":"2023-09-29T23:29:39.646512+0000","last_became_active":"2023-09-29T23:29:39.638232+0000","last_became_peered":"2023-09-29T23:29:39.638232+0000","last_unstale":"2023-09-29T23:29:39.646512+0000","last_undegraded":"2023-09-29T23:29:39.646512+0000","last_fullsized":"2023-09-29T23:29:39.646512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:45:45.006295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.598274+0000","last_change":"2023-09-29T23:29:35.095306+0000","last_active":"2023-09-29T23:29:39.598274+0000","last_peered":"2023-09-29T23:29:39.598274+0000","last_clean":"2023-09-29T23:29:39.598274+0000","last_became_active":"2023-09-29T23:29:35.093309+0000","last_became_peered":"2023-09-29T23:29:35.093309+0000","last_unstale":"2023-09-29T23:29:39.598274+0000","last_undegraded":"2023-09-29T23:29:39.598274+0000","last_fullsized":"2023-09-29T23:29:39.598274+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:10:21.305466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.595615+0000","last_change":"2023-09-29T23:29:39.261319+0000","last_active":"2023-09-29T23:29:39.595615+0000","last_peered":"2023-09-29T23:29:39.595615+0000","last_clean":"2023-09-29T23:29:39.595615+0000","last_became_active":"2023-09-29T23:29:39.260979+0000","last_became_peered":"2023-09-29T23:29:39.260979+0000","last_unstale":"2023-09-29T23:29:39.595615+0000","last_undegraded":"2023-09-29T23:29:39.595615+0000","last_fullsized":"2023-09-29T23:29:39.595615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:58:28.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625251+0000","last_change":"2023-09-29T23:29:39.620458+0000","last_active":"2023-09-29T23:29:39.625251+0000","last_peered":"2023-09-29T23:29:39.625251+0000","last_clean":"2023-09-29T23:29:39.625251+0000","last_became_active":"2023-09-29T23:29:39.620060+0000","last_became_peered":"2023-09-29T23:29:39.620060+0000","last_unstale":"2023-09-29T23:29:39.625251+0000","last_undegraded":"2023-09-29T23:29:39.625251+0000","last_fullsized":"2023-09-29T23:29:39.625251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:49.659581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617399+0000","last_change":"2023-09-29T23:29:35.099643+0000","last_active":"2023-09-29T23:29:39.617399+0000","last_peered":"2023-09-29T23:29:39.617399+0000","last_clean":"2023-09-29T23:29:39.617399+0000","last_became_active":"2023-09-29T23:29:35.099061+0000","last_became_peered":"2023-09-29T23:29:35.099061+0000","last_unstale":"2023-09-29T23:29:39.617399+0000","last_undegraded":"2023-09-29T23:29:39.617399+0000","last_fullsized":"2023-09-29T23:29:39.617399+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:30.050339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.599401+0000","last_change":"2023-09-29T23:29:39.259645+0000","last_active":"2023-09-29T23:29:39.599401+0000","last_peered":"2023-09-29T23:29:39.599401+0000","last_clean":"2023-09-29T23:29:39.599401+0000","last_became_active":"2023-09-29T23:29:39.258895+0000","last_became_peered":"2023-09-29T23:29:39.258895+0000","last_unstale":"2023-09-29T23:29:39.599401+0000","last_undegraded":"2023-09-29T23:29:39.599401+0000","last_fullsized":"2023-09-29T23:29:39.599401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:32:59.650256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609649+0000","last_change":"2023-09-29T23:29:39.608553+0000","last_active":"2023-09-29T23:29:39.609649+0000","last_peered":"2023-09-29T23:29:39.609649+0000","last_clean":"2023-09-29T23:29:39.609649+0000","last_became_active":"2023-09-29T23:29:39.608249+0000","last_became_peered":"2023-09-29T23:29:39.608249+0000","last_unstale":"2023-09-29T23:29:39.609649+0000","last_undegraded":"2023-09-29T23:29:39.609649+0000","last_fullsized":"2023-09-29T23:29:39.609649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:38:28.534318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.599877+0000","last_change":"2023-09-29T23:29:39.259646+0000","last_active":"2023-09-29T23:29:39.599877+0000","last_peered":"2023-09-29T23:29:39.599877+0000","last_clean":"2023-09-29T23:29:39.599877+0000","last_became_active":"2023-09-29T23:29:39.259285+0000","last_became_peered":"2023-09-29T23:29:39.259285+0000","last_unstale":"2023-09-29T23:29:39.599877+0000","last_undegraded":"2023-09-29T23:29:39.599877+0000","last_fullsized":"2023-09-29T23:29:39.599877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:47:29.475878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597541+0000","last_change":"2023-09-29T23:29:35.100113+0000","last_active":"2023-09-29T23:29:39.597541+0000","last_peered":"2023-09-29T23:29:39.597541+0000","last_clean":"2023-09-29T23:29:39.597541+0000","last_became_active":"2023-09-29T23:29:35.099578+0000","last_became_peered":"2023-09-29T23:29:35.099578+0000","last_unstale":"2023-09-29T23:29:39.597541+0000","last_undegraded":"2023-09-29T23:29:39.597541+0000","last_fullsized":"2023-09-29T23:29:39.597541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:57:39.508565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.640050+0000","last_change":"2023-09-29T23:29:39.637174+0000","last_active":"2023-09-29T23:29:39.640050+0000","last_peered":"2023-09-29T23:29:39.640050+0000","last_clean":"2023-09-29T23:29:39.640050+0000","last_became_active":"2023-09-29T23:29:39.636903+0000","last_became_peered":"2023-09-29T23:29:39.636903+0000","last_unstale":"2023-09-29T23:29:39.640050+0000","last_undegraded":"2023-09-29T23:29:39.640050+0000","last_fullsized":"2023-09-29T23:29:39.640050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:04:25.929196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597410+0000","last_change":"2023-09-29T23:29:39.271080+0000","last_active":"2023-09-29T23:29:39.597410+0000","last_peered":"2023-09-29T23:29:39.597410+0000","last_clean":"2023-09-29T23:29:39.597410+0000","last_became_active":"2023-09-29T23:29:39.270673+0000","last_became_peered":"2023-09-29T23:29:39.270673+0000","last_unstale":"2023-09-29T23:29:39.597410+0000","last_undegraded":"2023-09-29T23:29:39.597410+0000","last_fullsized":"2023-09-29T23:29:39.597410+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:28:04.709351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.628891+0000","last_change":"2023-09-29T23:29:35.099325+0000","last_active":"2023-09-29T23:29:39.628891+0000","last_peered":"2023-09-29T23:29:39.628891+0000","last_clean":"2023-09-29T23:29:39.628891+0000","last_became_active":"2023-09-29T23:29:35.098701+0000","last_became_peered":"2023-09-29T23:29:35.098701+0000","last_unstale":"2023-09-29T23:29:39.628891+0000","last_undegraded":"2023-09-29T23:29:39.628891+0000","last_fullsized":"2023-09-29T23:29:39.628891+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:35:07.769754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641024+0000","last_change":"2023-09-29T23:29:39.638042+0000","last_active":"2023-09-29T23:29:39.641024+0000","last_peered":"2023-09-29T23:29:39.641024+0000","last_clean":"2023-09-29T23:29:39.641024+0000","last_became_active":"2023-09-29T23:29:39.637714+0000","last_became_peered":"2023-09-29T23:29:39.637714+0000","last_unstale":"2023-09-29T23:29:39.641024+0000","last_undegraded":"2023-09-29T23:29:39.641024+0000","last_fullsized":"2023-09-29T23:29:39.641024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:47:58.354132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.644775+0000","last_change":"2023-09-29T23:29:39.269267+0000","last_active":"2023-09-29T23:29:39.644775+0000","last_peered":"2023-09-29T23:29:39.644775+0000","last_clean":"2023-09-29T23:29:39.644775+0000","last_became_active":"2023-09-29T23:29:39.268949+0000","last_became_peered":"2023-09-29T23:29:39.268949+0000","last_unstale":"2023-09-29T23:29:39.644775+0000","last_undegraded":"2023-09-29T23:29:39.644775+0000","last_fullsized":"2023-09-29T23:29:39.644775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:07:08.485719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597483+0000","last_change":"2023-09-29T23:29:35.096994+0000","last_active":"2023-09-29T23:29:39.597483+0000","last_peered":"2023-09-29T23:29:39.597483+0000","last_clean":"2023-09-29T23:29:39.597483+0000","last_became_active":"2023-09-29T23:29:35.096568+0000","last_became_peered":"2023-09-29T23:29:35.096568+0000","last_unstale":"2023-09-29T23:29:39.597483+0000","last_undegraded":"2023-09-29T23:29:39.597483+0000","last_fullsized":"2023-09-29T23:29:39.597483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:22:06.698301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645797+0000","last_change":"2023-09-29T23:29:39.638385+0000","last_active":"2023-09-29T23:29:39.645797+0000","last_peered":"2023-09-29T23:29:39.645797+0000","last_clean":"2023-09-29T23:29:39.645797+0000","last_became_active":"2023-09-29T23:29:39.634957+0000","last_became_peered":"2023-09-29T23:29:39.634957+0000","last_unstale":"2023-09-29T23:29:39.645797+0000","last_undegraded":"2023-09-29T23:29:39.645797+0000","last_fullsized":"2023-09-29T23:29:39.645797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:13:11.017795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-29T23:29:41.577250+0000","last_change":"2023-09-29T23:29:39.271004+0000","last_active":"2023-09-29T23:29:41.577250+0000","last_peered":"2023-09-29T23:29:41.577250+0000","last_clean":"2023-09-29T23:29:41.577250+0000","last_became_active":"2023-09-29T23:29:39.270721+0000","last_became_peered":"2023-09-29T23:29:39.270721+0000","last_unstale":"2023-09-29T23:29:41.577250+0000","last_undegraded":"2023-09-29T23:29:41.577250+0000","last_fullsized":"2023-09-29T23:29:41.577250+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:18:38.314574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620608+0000","last_change":"2023-09-29T23:29:35.776424+0000","last_active":"2023-09-29T23:29:39.620608+0000","last_peered":"2023-09-29T23:29:39.620608+0000","last_clean":"2023-09-29T23:29:39.620608+0000","last_became_active":"2023-09-29T23:29:35.775625+0000","last_became_peered":"2023-09-29T23:29:35.775625+0000","last_unstale":"2023-09-29T23:29:39.620608+0000","last_undegraded":"2023-09-29T23:29:39.620608+0000","last_fullsized":"2023-09-29T23:29:39.620608+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:46.078517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.627165+0000","last_change":"2023-09-29T23:29:39.624813+0000","last_active":"2023-09-29T23:29:39.627165+0000","last_peered":"2023-09-29T23:29:39.627165+0000","last_clean":"2023-09-29T23:29:39.627165+0000","last_became_active":"2023-09-29T23:29:39.624521+0000","last_became_peered":"2023-09-29T23:29:39.624521+0000","last_unstale":"2023-09-29T23:29:39.627165+0000","last_undegraded":"2023-09-29T23:29:39.627165+0000","last_fullsized":"2023-09-29T23:29:39.627165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:38:28.534318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.639914+0000","last_change":"2023-09-29T23:29:39.635422+0000","last_active":"2023-09-29T23:29:39.639914+0000","last_peered":"2023-09-29T23:29:39.639914+0000","last_clean":"2023-09-29T23:29:39.639914+0000","last_became_active":"2023-09-29T23:29:39.635139+0000","last_became_peered":"2023-09-29T23:29:39.635139+0000","last_unstale":"2023-09-29T23:29:39.639914+0000","last_undegraded":"2023-09-29T23:29:39.639914+0000","last_fullsized":"2023-09-29T23:29:39.639914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:04:09.469416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"18'6","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647558+0000","last_change":"2023-09-29T23:29:39.271958+0000","last_active":"2023-09-29T23:29:39.647558+0000","last_peered":"2023-09-29T23:29:39.647558+0000","last_clean":"2023-09-29T23:29:39.647558+0000","last_became_active":"2023-09-29T23:29:39.271601+0000","last_became_peered":"2023-09-29T23:29:39.271601+0000","last_unstale":"2023-09-29T23:29:39.647558+0000","last_undegraded":"2023-09-29T23:29:39.647558+0000","last_fullsized":"2023-09-29T23:29:39.647558+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:20:36.278014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637865+0000","last_change":"2023-09-29T23:29:39.633505+0000","last_active":"2023-09-29T23:29:39.637865+0000","last_peered":"2023-09-29T23:29:39.637865+0000","last_clean":"2023-09-29T23:29:39.637865+0000","last_became_active":"2023-09-29T23:29:39.633253+0000","last_became_peered":"2023-09-29T23:29:39.633253+0000","last_unstale":"2023-09-29T23:29:39.637865+0000","last_undegraded":"2023-09-29T23:29:39.637865+0000","last_fullsized":"2023-09-29T23:29:39.637865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:53:25.888059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"18'3","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.633016+0000","last_change":"2023-09-29T23:29:39.268518+0000","last_active":"2023-09-29T23:29:39.633016+0000","last_peered":"2023-09-29T23:29:39.633016+0000","last_clean":"2023-09-29T23:29:39.633016+0000","last_became_active":"2023-09-29T23:29:39.267411+0000","last_became_peered":"2023-09-29T23:29:39.267411+0000","last_unstale":"2023-09-29T23:29:39.633016+0000","last_undegraded":"2023-09-29T23:29:39.633016+0000","last_fullsized":"2023-09-29T23:29:39.633016+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:18:38.314574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.601527+0000","last_change":"2023-09-29T23:29:39.267677+0000","last_active":"2023-09-29T23:29:39.601527+0000","last_peered":"2023-09-29T23:29:39.601527+0000","last_clean":"2023-09-29T23:29:39.601527+0000","last_became_active":"2023-09-29T23:29:39.267374+0000","last_became_peered":"2023-09-29T23:29:39.267374+0000","last_unstale":"2023-09-29T23:29:39.601527+0000","last_undegraded":"2023-09-29T23:29:39.601527+0000","last_fullsized":"2023-09-29T23:29:39.601527+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:23:58.356932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637457+0000","last_change":"2023-09-29T23:29:39.633662+0000","last_active":"2023-09-29T23:29:39.637457+0000","last_peered":"2023-09-29T23:29:39.637457+0000","last_clean":"2023-09-29T23:29:39.637457+0000","last_became_active":"2023-09-29T23:29:39.633327+0000","last_became_peered":"2023-09-29T23:29:39.633327+0000","last_unstale":"2023-09-29T23:29:39.637457+0000","last_undegraded":"2023-09-29T23:29:39.637457+0000","last_fullsized":"2023-09-29T23:29:39.637457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:59:22.894223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.618457+0000","last_change":"2023-09-29T23:29:39.260818+0000","last_active":"2023-09-29T23:29:39.618457+0000","last_peered":"2023-09-29T23:29:39.618457+0000","last_clean":"2023-09-29T23:29:39.618457+0000","last_became_active":"2023-09-29T23:29:39.260529+0000","last_became_peered":"2023-09-29T23:29:39.260529+0000","last_unstale":"2023-09-29T23:29:39.618457+0000","last_undegraded":"2023-09-29T23:29:39.618457+0000","last_fullsized":"2023-09-29T23:29:39.618457+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:47:31.314319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645754+0000","last_change":"2023-09-29T23:29:39.635312+0000","last_active":"2023-09-29T23:29:39.645754+0000","last_peered":"2023-09-29T23:29:39.645754+0000","last_clean":"2023-09-29T23:29:39.645754+0000","last_became_active":"2023-09-29T23:29:39.633112+0000","last_became_peered":"2023-09-29T23:29:39.633112+0000","last_unstale":"2023-09-29T23:29:39.645754+0000","last_undegraded":"2023-09-29T23:29:39.645754+0000","last_fullsized":"2023-09-29T23:29:39.645754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:55:52.762563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645311+0000","last_change":"2023-09-29T23:29:39.635051+0000","last_active":"2023-09-29T23:29:39.645311+0000","last_peered":"2023-09-29T23:29:39.645311+0000","last_clean":"2023-09-29T23:29:39.645311+0000","last_became_active":"2023-09-29T23:29:39.632920+0000","last_became_peered":"2023-09-29T23:29:39.632920+0000","last_unstale":"2023-09-29T23:29:39.645311+0000","last_undegraded":"2023-09-29T23:29:39.645311+0000","last_fullsized":"2023-09-29T23:29:39.645311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:55:37.158884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617319+0000","last_change":"2023-09-29T23:29:39.268725+0000","last_active":"2023-09-29T23:29:39.617319+0000","last_peered":"2023-09-29T23:29:39.617319+0000","last_clean":"2023-09-29T23:29:39.617319+0000","last_became_active":"2023-09-29T23:29:39.268360+0000","last_became_peered":"2023-09-29T23:29:39.268360+0000","last_unstale":"2023-09-29T23:29:39.617319+0000","last_undegraded":"2023-09-29T23:29:39.617319+0000","last_fullsized":"2023-09-29T23:29:39.617319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:23:58.356932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617990+0000","last_change":"2023-09-29T23:29:39.616291+0000","last_active":"2023-09-29T23:29:39.617990+0000","last_peered":"2023-09-29T23:29:39.617990+0000","last_clean":"2023-09-29T23:29:39.617990+0000","last_became_active":"2023-09-29T23:29:39.615966+0000","last_became_peered":"2023-09-29T23:29:39.615966+0000","last_unstale":"2023-09-29T23:29:39.617990+0000","last_undegraded":"2023-09-29T23:29:39.617990+0000","last_fullsized":"2023-09-29T23:29:39.617990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:07:09.489477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.605702+0000","last_change":"2023-09-29T23:29:39.259196+0000","last_active":"2023-09-29T23:29:39.605702+0000","last_peered":"2023-09-29T23:29:39.605702+0000","last_clean":"2023-09-29T23:29:39.605702+0000","last_became_active":"2023-09-29T23:29:39.258785+0000","last_became_peered":"2023-09-29T23:29:39.258785+0000","last_unstale":"2023-09-29T23:29:39.605702+0000","last_undegraded":"2023-09-29T23:29:39.605702+0000","last_fullsized":"2023-09-29T23:29:39.605702+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:23:58.356932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647103+0000","last_change":"2023-09-29T23:29:39.635946+0000","last_active":"2023-09-29T23:29:39.647103+0000","last_peered":"2023-09-29T23:29:39.647103+0000","last_clean":"2023-09-29T23:29:39.647103+0000","last_became_active":"2023-09-29T23:29:39.634129+0000","last_became_peered":"2023-09-29T23:29:39.634129+0000","last_unstale":"2023-09-29T23:29:39.647103+0000","last_undegraded":"2023-09-29T23:29:39.647103+0000","last_fullsized":"2023-09-29T23:29:39.647103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:31:36.612565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629612+0000","last_change":"2023-09-29T23:29:39.252290+0000","last_active":"2023-09-29T23:29:39.629612+0000","last_peered":"2023-09-29T23:29:39.629612+0000","last_clean":"2023-09-29T23:29:39.629612+0000","last_became_active":"2023-09-29T23:29:39.246835+0000","last_became_peered":"2023-09-29T23:29:39.246835+0000","last_unstale":"2023-09-29T23:29:39.629612+0000","last_undegraded":"2023-09-29T23:29:39.629612+0000","last_fullsized":"2023-09-29T23:29:39.629612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:09:46.708805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609639+0000","last_change":"2023-09-29T23:29:39.608519+0000","last_active":"2023-09-29T23:29:39.609639+0000","last_peered":"2023-09-29T23:29:39.609639+0000","last_clean":"2023-09-29T23:29:39.609639+0000","last_became_active":"2023-09-29T23:29:39.608334+0000","last_became_peered":"2023-09-29T23:29:39.608334+0000","last_unstale":"2023-09-29T23:29:39.609639+0000","last_undegraded":"2023-09-29T23:29:39.609639+0000","last_fullsized":"2023-09-29T23:29:39.609639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:31.379677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.632767+0000","last_change":"2023-09-29T23:29:39.270477+0000","last_active":"2023-09-29T23:29:39.632767+0000","last_peered":"2023-09-29T23:29:39.632767+0000","last_clean":"2023-09-29T23:29:39.632767+0000","last_became_active":"2023-09-29T23:29:39.270178+0000","last_became_peered":"2023-09-29T23:29:39.270178+0000","last_unstale":"2023-09-29T23:29:39.632767+0000","last_undegraded":"2023-09-29T23:29:39.632767+0000","last_fullsized":"2023-09-29T23:29:39.632767+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:05:51.179867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.608417+0000","last_change":"2023-09-29T23:29:39.605080+0000","last_active":"2023-09-29T23:29:39.608417+0000","last_peered":"2023-09-29T23:29:39.608417+0000","last_clean":"2023-09-29T23:29:39.608417+0000","last_became_active":"2023-09-29T23:29:39.604701+0000","last_became_peered":"2023-09-29T23:29:39.604701+0000","last_unstale":"2023-09-29T23:29:39.608417+0000","last_undegraded":"2023-09-29T23:29:39.608417+0000","last_fullsized":"2023-09-29T23:29:39.608417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:49.659581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645866+0000","last_change":"2023-09-29T23:29:39.259176+0000","last_active":"2023-09-29T23:29:39.645866+0000","last_peered":"2023-09-29T23:29:39.645866+0000","last_clean":"2023-09-29T23:29:39.645866+0000","last_became_active":"2023-09-29T23:29:39.258811+0000","last_became_peered":"2023-09-29T23:29:39.258811+0000","last_unstale":"2023-09-29T23:29:39.645866+0000","last_undegraded":"2023-09-29T23:29:39.645866+0000","last_fullsized":"2023-09-29T23:29:39.645866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:38:39.409933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.608584+0000","last_change":"2023-09-29T23:29:39.606077+0000","last_active":"2023-09-29T23:29:39.608584+0000","last_peered":"2023-09-29T23:29:39.608584+0000","last_clean":"2023-09-29T23:29:39.608584+0000","last_became_active":"2023-09-29T23:29:39.605747+0000","last_became_peered":"2023-09-29T23:29:39.605747+0000","last_unstale":"2023-09-29T23:29:39.608584+0000","last_undegraded":"2023-09-29T23:29:39.608584+0000","last_fullsized":"2023-09-29T23:29:39.608584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:07:09.489477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.644653+0000","last_change":"2023-09-29T23:29:39.273703+0000","last_active":"2023-09-29T23:29:39.644653+0000","last_peered":"2023-09-29T23:29:39.644653+0000","last_clean":"2023-09-29T23:29:39.644653+0000","last_became_active":"2023-09-29T23:29:39.273385+0000","last_became_peered":"2023-09-29T23:29:39.273385+0000","last_unstale":"2023-09-29T23:29:39.644653+0000","last_undegraded":"2023-09-29T23:29:39.644653+0000","last_fullsized":"2023-09-29T23:29:39.644653+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:04:08.465658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645244+0000","last_change":"2023-09-29T23:29:39.634607+0000","last_active":"2023-09-29T23:29:39.645244+0000","last_peered":"2023-09-29T23:29:39.645244+0000","last_clean":"2023-09-29T23:29:39.645244+0000","last_became_active":"2023-09-29T23:29:39.633666+0000","last_became_peered":"2023-09-29T23:29:39.633666+0000","last_unstale":"2023-09-29T23:29:39.645244+0000","last_undegraded":"2023-09-29T23:29:39.645244+0000","last_fullsized":"2023-09-29T23:29:39.645244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:19:23.819855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609036+0000","last_change":"2023-09-29T23:29:39.271329+0000","last_active":"2023-09-29T23:29:39.609036+0000","last_peered":"2023-09-29T23:29:39.609036+0000","last_clean":"2023-09-29T23:29:39.609036+0000","last_became_active":"2023-09-29T23:29:39.270902+0000","last_became_peered":"2023-09-29T23:29:39.270902+0000","last_unstale":"2023-09-29T23:29:39.609036+0000","last_undegraded":"2023-09-29T23:29:39.609036+0000","last_fullsized":"2023-09-29T23:29:39.609036+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:43:32.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609490+0000","last_change":"2023-09-29T23:29:39.605447+0000","last_active":"2023-09-29T23:29:39.609490+0000","last_peered":"2023-09-29T23:29:39.609490+0000","last_clean":"2023-09-29T23:29:39.609490+0000","last_became_active":"2023-09-29T23:29:39.604800+0000","last_became_peered":"2023-09-29T23:29:39.604800+0000","last_unstale":"2023-09-29T23:29:39.609490+0000","last_undegraded":"2023-09-29T23:29:39.609490+0000","last_fullsized":"2023-09-29T23:29:39.609490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:58:35.776573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"18'4","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.631433+0000","last_change":"2023-09-29T23:29:39.251909+0000","last_active":"2023-09-29T23:29:39.631433+0000","last_peered":"2023-09-29T23:29:39.631433+0000","last_clean":"2023-09-29T23:29:39.631433+0000","last_became_active":"2023-09-29T23:29:39.251464+0000","last_became_peered":"2023-09-29T23:29:39.251464+0000","last_unstale":"2023-09-29T23:29:39.631433+0000","last_undegraded":"2023-09-29T23:29:39.631433+0000","last_fullsized":"2023-09-29T23:29:39.631433+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:58:28.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705672,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12684,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":12480,"kb_avail":94359156,"statfs":{"total":96636764160,"available":96623775744,"internally_reserved":0,"allocated":200704,"data_stored":62103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12779520},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13132,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358708,"statfs":{"total":96636764160,"available":96623316992,"internally_reserved":0,"allocated":266240,"data_stored":652423,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13120,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358720,"statfs":{"total":96636764160,"available":96623329280,"internally_reserved":0,"allocated":253952,"data_stored":651387,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8440,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363400,"statfs":{"total":96636764160,"available":96628121600,"internally_reserved":0,"allocated":180224,"data_stored":60927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T23:29:50.429 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-29T23:29:50.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.532+0000 7f2015930700 1 Processor -- start 2023-09-29T23:29:50.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.532+0000 7f2015930700 1 -- start start 2023-09-29T23:29:50.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 0x7f20101442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f201006da00 0x7f2010143820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f201006acd0 0x7f2010135f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2010143e60 con 0x7f201006acd0 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20100b3510 con 0x7f201006da00 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201006b3b0 con 0x7f20100b3140 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f201006acd0 0x7f2010135f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200effd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 0x7f20101442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f201006acd0 0x7f2010135f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34700/0 (socket says 172.21.15.143:34700) 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200f7fe700 1 -- 172.21.15.143:0/734302303 learned_addr learned my addr 172.21.15.143:0/734302303 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200effd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 0x7f20101442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60540/0 (socket says 172.21.15.143:60540) 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200e7fc700 1 --2- 172.21.15.143:0/734302303 >> v2:172.21.15.156:3300/0 conn(0x7f201006da00 0x7f2010143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200effd700 1 -- 172.21.15.143:0/734302303 >> v2:172.21.15.156:3300/0 conn(0x7f201006da00 msgr2=0x7f2010143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200effd700 1 --2- 172.21.15.143:0/734302303 >> v2:172.21.15.156:3300/0 conn(0x7f201006da00 0x7f2010143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200effd700 1 -- 172.21.15.143:0/734302303 >> v2:172.21.15.143:3300/0 conn(0x7f201006acd0 msgr2=0x7f2010135f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:50.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200effd700 1 --2- 172.21.15.143:0/734302303 >> v2:172.21.15.143:3300/0 conn(0x7f201006acd0 0x7f2010135f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200effd700 1 -- 172.21.15.143:0/734302303 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201006b0f0 con 0x7f20100b3140 2023-09-29T23:29:50.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200f7fe700 1 --2- 172.21.15.143:0/734302303 >> v2:172.21.15.143:3300/0 conn(0x7f201006acd0 0x7f2010135f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:29:50.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200effd700 1 --2- 172.21.15.143:0/734302303 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 0x7f20101442f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f201006e560 tx=0x7f20040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27c8e0654ae5384d server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:50.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200dffb700 1 -- 172.21.15.143:0/734302303 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004011630 con 0x7f20100b3140 2023-09-29T23:29:50.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200dffb700 1 -- 172.21.15.143:0/734302303 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20040117b0 con 0x7f20100b3140 2023-09-29T23:29:50.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f200dffb700 1 -- 172.21.15.143:0/734302303 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004011b70 con 0x7f20100b3140 2023-09-29T23:29:50.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 -- 172.21.15.143:0/734302303 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 msgr2=0x7f20101442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 --2- 172.21.15.143:0/734302303 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 0x7f20101442f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f201006e560 tx=0x7f20040091a0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 -- 172.21.15.143:0/734302303 shutdown_connections 2023-09-29T23:29:50.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 --2- 172.21.15.143:0/734302303 >> v2:172.21.15.143:3300/0 conn(0x7f201006acd0 0x7f2010135f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 --2- 172.21.15.143:0/734302303 >> v2:172.21.15.156:3300/0 conn(0x7f201006da00 0x7f2010143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 --2- 172.21.15.143:0/734302303 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 0x7f20101442f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 -- 172.21.15.143:0/734302303 >> 172.21.15.143:0/734302303 conn(0x7f2010067dc0 msgr2=0x7f20100b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:50.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 -- 172.21.15.143:0/734302303 shutdown_connections 2023-09-29T23:29:50.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.536+0000 7f2015930700 1 -- 172.21.15.143:0/734302303 wait complete. 2023-09-29T23:29:50.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f2015930700 1 Processor -- start 2023-09-29T23:29:50.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f2015930700 1 -- start start 2023-09-29T23:29:50.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f2015930700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f201006acd0 0x7f201014f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f2015930700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201006da00 0x7f2010139a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f2015930700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 0x7f2010138140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f2015930700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010133100 con 0x7f201006da00 2023-09-29T23:29:50.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f2015930700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010139000 con 0x7f201006acd0 2023-09-29T23:29:50.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f2015930700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20101392e0 con 0x7f20100b3140 2023-09-29T23:29:50.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f200e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201006da00 0x7f2010139a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f200e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201006da00 0x7f2010139a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34716/0 (socket says 172.21.15.143:34716) 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f200f7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 0x7f2010138140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f200e7fc700 1 -- 172.21.15.143:0/3582851847 learned_addr learned my addr 172.21.15.143:0/3582851847 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f200effd700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f201006acd0 0x7f201014f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f200f7fe700 1 -- 172.21.15.143:0/3582851847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f201006acd0 msgr2=0x7f201014f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f200f7fe700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f201006acd0 0x7f201014f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f200f7fe700 1 -- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201006da00 msgr2=0x7f2010139a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f200f7fe700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201006da00 0x7f2010139a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f200f7fe700 1 -- 172.21.15.143:0/3582851847 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000094f0 con 0x7f20100b3140 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f200f7fe700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 0x7f2010138140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f200000ea20 tx=0x7f2000009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f1ff6ffd700 1 -- 172.21.15.143:0/3582851847 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200000ee60 con 0x7f20100b3140 2023-09-29T23:29:50.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f1ff6ffd700 1 -- 172.21.15.143:0/3582851847 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200000c850 con 0x7f20100b3140 2023-09-29T23:29:50.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f1ff6ffd700 1 -- 172.21.15.143:0/3582851847 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200000cbd0 con 0x7f20100b3140 2023-09-29T23:29:50.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f2015930700 1 -- 172.21.15.143:0/3582851847 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2004009420 con 0x7f20100b3140 2023-09-29T23:29:50.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.540+0000 7f2015930700 1 -- 172.21.15.143:0/3582851847 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2010150300 con 0x7f20100b3140 2023-09-29T23:29:50.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.544+0000 7f2015930700 1 -- 172.21.15.143:0/3582851847 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201014fe30 con 0x7f20100b3140 2023-09-29T23:29:50.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.544+0000 7f1ff6ffd700 1 -- 172.21.15.143:0/3582851847 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f2000013470 con 0x7f20100b3140 2023-09-29T23:29:50.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.544+0000 7f1ff6ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:50.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.544+0000 7f1ff6ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:50.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.544+0000 7f1ff6ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:50.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.544+0000 7f1ff6ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:50.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.544+0000 7f1ff6ffd700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1ffc08e830 0x7f1ffc090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.544+0000 7f1ff6ffd700 1 -- 172.21.15.143:0/3582851847 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f20000b4b30 con 0x7f20100b3140 2023-09-29T23:29:50.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.544+0000 7f200effd700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1ffc08e830 0x7f1ffc090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.544+0000 7f200effd700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1ffc08e830 0x7f1ffc090ce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f201006e560 tx=0x7f2004017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:50.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.552+0000 7f1ff6ffd700 1 -- 172.21.15.143:0/3582851847 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f20000855f0 con 0x7f20100b3140 2023-09-29T23:29:50.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.708+0000 7f2015930700 20 mgrc start_command cmd: [{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}] 2023-09-29T23:29:50.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.708+0000 7f2015930700 1 -- 172.21.15.143:0/3582851847 --> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2010133270 con 0x7f1ffc08e830 2023-09-29T23:29:50.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.724+0000 7f1ff6ffd700 1 -- 172.21.15.143:0/3582851847 <== mgr.4101 v2:172.21.15.143:6808/27602 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+269431 (secure 0 0 0) 0x7f2010133270 con 0x7f1ffc08e830 2023-09-29T23:29:50.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.724+0000 7f1ff6ffd700 20 mgrc handle_command_reply tid 0 r 0 2023-09-29T23:29:50.730 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:50.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 10 mgrc shutdown 2023-09-29T23:29:50.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 -- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1ffc08e830 msgr2=0x7f1ffc090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.732 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1ffc08e830 0x7f1ffc090ce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f201006e560 tx=0x7f2004017000 comp rx=0 tx=0).stop 2023-09-29T23:29:50.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 -- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 msgr2=0x7f2010138140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 0x7f2010138140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f200000ea20 tx=0x7f2000009250 comp rx=0 tx=0).stop 2023-09-29T23:29:50.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 -- 172.21.15.143:0/3582851847 shutdown_connections 2023-09-29T23:29:50.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f1ffc08e830 0x7f1ffc090ce0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f20100b3140 0x7f2010138140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f201006da00 0x7f2010139a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 --2- 172.21.15.143:0/3582851847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f201006acd0 0x7f201014f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 -- 172.21.15.143:0/3582851847 >> 172.21.15.143:0/3582851847 conn(0x7f2010067dc0 msgr2=0x7f2010142310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:50.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 -- 172.21.15.143:0/3582851847 shutdown_connections 2023-09-29T23:29:50.733 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.728+0000 7f2015930700 1 -- 172.21.15.143:0/3582851847 wait complete. 2023-09-29T23:29:50.734 INFO:teuthology.orchestra.run.smithi143.stderr:dumped all 2023-09-29T23:29:50.746 INFO:teuthology.orchestra.run.smithi143.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-09-29T23:29:50.465526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":47376,"kb_used_data":880,"kb_used_omap":0,"kb_used_meta":46464,"kb_avail":377439984,"statfs":{"total":386547056640,"available":386498543616,"internally_reserved":0,"allocated":901120,"data_stored":1426840,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":47579136},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003362"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.639872+0000","last_change":"2023-09-29T23:29:39.634761+0000","last_active":"2023-09-29T23:29:39.639872+0000","last_peered":"2023-09-29T23:29:39.639872+0000","last_clean":"2023-09-29T23:29:39.639872+0000","last_became_active":"2023-09-29T23:29:39.634501+0000","last_became_peered":"2023-09-29T23:29:39.634501+0000","last_unstale":"2023-09-29T23:29:39.639872+0000","last_undegraded":"2023-09-29T23:29:39.639872+0000","last_fullsized":"2023-09-29T23:29:39.639872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:20:37.281771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620836+0000","last_change":"2023-09-29T23:29:39.615851+0000","last_active":"2023-09-29T23:29:39.620836+0000","last_peered":"2023-09-29T23:29:39.620836+0000","last_clean":"2023-09-29T23:29:39.620836+0000","last_became_active":"2023-09-29T23:29:39.615521+0000","last_became_peered":"2023-09-29T23:29:39.615521+0000","last_unstale":"2023-09-29T23:29:39.620836+0000","last_undegraded":"2023-09-29T23:29:39.620836+0000","last_fullsized":"2023-09-29T23:29:39.620836+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:17:42.897947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647121+0000","last_change":"2023-09-29T23:29:39.636680+0000","last_active":"2023-09-29T23:29:39.647121+0000","last_peered":"2023-09-29T23:29:39.647121+0000","last_clean":"2023-09-29T23:29:39.647121+0000","last_became_active":"2023-09-29T23:29:39.634721+0000","last_became_peered":"2023-09-29T23:29:39.634721+0000","last_unstale":"2023-09-29T23:29:39.647121+0000","last_undegraded":"2023-09-29T23:29:39.647121+0000","last_fullsized":"2023-09-29T23:29:39.647121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:21:36.503704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625198+0000","last_change":"2023-09-29T23:29:39.612565+0000","last_active":"2023-09-29T23:29:39.625198+0000","last_peered":"2023-09-29T23:29:39.625198+0000","last_clean":"2023-09-29T23:29:39.625198+0000","last_became_active":"2023-09-29T23:29:39.612183+0000","last_became_peered":"2023-09-29T23:29:39.612183+0000","last_unstale":"2023-09-29T23:29:39.625198+0000","last_undegraded":"2023-09-29T23:29:39.625198+0000","last_fullsized":"2023-09-29T23:29:39.625198+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:42.881623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645186+0000","last_change":"2023-09-29T23:29:39.633553+0000","last_active":"2023-09-29T23:29:39.645186+0000","last_peered":"2023-09-29T23:29:39.645186+0000","last_clean":"2023-09-29T23:29:39.645186+0000","last_became_active":"2023-09-29T23:29:39.633127+0000","last_became_peered":"2023-09-29T23:29:39.633127+0000","last_unstale":"2023-09-29T23:29:39.645186+0000","last_undegraded":"2023-09-29T23:29:39.645186+0000","last_fullsized":"2023-09-29T23:29:39.645186+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:47:58.354132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645627+0000","last_change":"2023-09-29T23:29:39.636332+0000","last_active":"2023-09-29T23:29:39.645627+0000","last_peered":"2023-09-29T23:29:39.645627+0000","last_clean":"2023-09-29T23:29:39.645627+0000","last_became_active":"2023-09-29T23:29:39.636025+0000","last_became_peered":"2023-09-29T23:29:39.636025+0000","last_unstale":"2023-09-29T23:29:39.645627+0000","last_undegraded":"2023-09-29T23:29:39.645627+0000","last_fullsized":"2023-09-29T23:29:39.645627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:24:33.457443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645224+0000","last_change":"2023-09-29T23:29:39.634398+0000","last_active":"2023-09-29T23:29:39.645224+0000","last_peered":"2023-09-29T23:29:39.645224+0000","last_clean":"2023-09-29T23:29:39.645224+0000","last_became_active":"2023-09-29T23:29:39.633340+0000","last_became_peered":"2023-09-29T23:29:39.633340+0000","last_unstale":"2023-09-29T23:29:39.645224+0000","last_undegraded":"2023-09-29T23:29:39.645224+0000","last_fullsized":"2023-09-29T23:29:39.645224+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:33:09.568111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629860+0000","last_change":"2023-09-29T23:29:39.618059+0000","last_active":"2023-09-29T23:29:39.629860+0000","last_peered":"2023-09-29T23:29:39.629860+0000","last_clean":"2023-09-29T23:29:39.629860+0000","last_became_active":"2023-09-29T23:29:39.617767+0000","last_became_peered":"2023-09-29T23:29:39.617767+0000","last_unstale":"2023-09-29T23:29:39.629860+0000","last_undegraded":"2023-09-29T23:29:39.629860+0000","last_fullsized":"2023-09-29T23:29:39.629860+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:55:52.762563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.630737+0000","last_change":"2023-09-29T23:29:39.626861+0000","last_active":"2023-09-29T23:29:39.630737+0000","last_peered":"2023-09-29T23:29:39.630737+0000","last_clean":"2023-09-29T23:29:39.630737+0000","last_became_active":"2023-09-29T23:29:39.626482+0000","last_became_peered":"2023-09-29T23:29:39.626482+0000","last_unstale":"2023-09-29T23:29:39.630737+0000","last_undegraded":"2023-09-29T23:29:39.630737+0000","last_fullsized":"2023-09-29T23:29:39.630737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:48.157072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641519+0000","last_change":"2023-09-29T23:29:39.640158+0000","last_active":"2023-09-29T23:29:39.641519+0000","last_peered":"2023-09-29T23:29:39.641519+0000","last_clean":"2023-09-29T23:29:39.641519+0000","last_became_active":"2023-09-29T23:29:39.639864+0000","last_became_peered":"2023-09-29T23:29:39.639864+0000","last_unstale":"2023-09-29T23:29:39.641519+0000","last_undegraded":"2023-09-29T23:29:39.641519+0000","last_fullsized":"2023-09-29T23:29:39.641519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:46:27.931849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625089+0000","last_change":"2023-09-29T23:29:39.612352+0000","last_active":"2023-09-29T23:29:39.625089+0000","last_peered":"2023-09-29T23:29:39.625089+0000","last_clean":"2023-09-29T23:29:39.625089+0000","last_became_active":"2023-09-29T23:29:39.611961+0000","last_became_peered":"2023-09-29T23:29:39.611961+0000","last_unstale":"2023-09-29T23:29:39.625089+0000","last_undegraded":"2023-09-29T23:29:39.625089+0000","last_fullsized":"2023-09-29T23:29:39.625089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:46:32.838187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638115+0000","last_change":"2023-09-29T23:29:39.632780+0000","last_active":"2023-09-29T23:29:39.638115+0000","last_peered":"2023-09-29T23:29:39.638115+0000","last_clean":"2023-09-29T23:29:39.638115+0000","last_became_active":"2023-09-29T23:29:39.632492+0000","last_became_peered":"2023-09-29T23:29:39.632492+0000","last_unstale":"2023-09-29T23:29:39.638115+0000","last_undegraded":"2023-09-29T23:29:39.638115+0000","last_fullsized":"2023-09-29T23:29:39.638115+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:14:21.471042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.636401+0000","last_change":"2023-09-29T23:29:39.633421+0000","last_active":"2023-09-29T23:29:39.636401+0000","last_peered":"2023-09-29T23:29:39.636401+0000","last_clean":"2023-09-29T23:29:39.636401+0000","last_became_active":"2023-09-29T23:29:39.633127+0000","last_became_peered":"2023-09-29T23:29:39.633127+0000","last_unstale":"2023-09-29T23:29:39.636401+0000","last_undegraded":"2023-09-29T23:29:39.636401+0000","last_fullsized":"2023-09-29T23:29:39.636401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:58:35.776573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629920+0000","last_change":"2023-09-29T23:29:39.621931+0000","last_active":"2023-09-29T23:29:39.629920+0000","last_peered":"2023-09-29T23:29:39.629920+0000","last_clean":"2023-09-29T23:29:39.629920+0000","last_became_active":"2023-09-29T23:29:39.621616+0000","last_became_peered":"2023-09-29T23:29:39.621616+0000","last_unstale":"2023-09-29T23:29:39.629920+0000","last_undegraded":"2023-09-29T23:29:39.629920+0000","last_fullsized":"2023-09-29T23:29:39.629920+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:53:43.483894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629993+0000","last_change":"2023-09-29T23:29:39.622200+0000","last_active":"2023-09-29T23:29:39.629993+0000","last_peered":"2023-09-29T23:29:39.629993+0000","last_clean":"2023-09-29T23:29:39.629993+0000","last_became_active":"2023-09-29T23:29:39.621385+0000","last_became_peered":"2023-09-29T23:29:39.621385+0000","last_unstale":"2023-09-29T23:29:39.629993+0000","last_undegraded":"2023-09-29T23:29:39.629993+0000","last_fullsized":"2023-09-29T23:29:39.629993+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:17:19.555819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645817+0000","last_change":"2023-09-29T23:29:39.638607+0000","last_active":"2023-09-29T23:29:39.645817+0000","last_peered":"2023-09-29T23:29:39.645817+0000","last_clean":"2023-09-29T23:29:39.645817+0000","last_became_active":"2023-09-29T23:29:39.635921+0000","last_became_peered":"2023-09-29T23:29:39.635921+0000","last_unstale":"2023-09-29T23:29:39.645817+0000","last_undegraded":"2023-09-29T23:29:39.645817+0000","last_fullsized":"2023-09-29T23:29:39.645817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:27:52.497036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.631406+0000","last_change":"2023-09-29T23:29:39.622234+0000","last_active":"2023-09-29T23:29:39.631406+0000","last_peered":"2023-09-29T23:29:39.631406+0000","last_clean":"2023-09-29T23:29:39.631406+0000","last_became_active":"2023-09-29T23:29:39.621861+0000","last_became_peered":"2023-09-29T23:29:39.621861+0000","last_unstale":"2023-09-29T23:29:39.631406+0000","last_undegraded":"2023-09-29T23:29:39.631406+0000","last_fullsized":"2023-09-29T23:29:39.631406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:54:29.811464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.646545+0000","last_change":"2023-09-29T23:29:39.639501+0000","last_active":"2023-09-29T23:29:39.646545+0000","last_peered":"2023-09-29T23:29:39.646545+0000","last_clean":"2023-09-29T23:29:39.646545+0000","last_became_active":"2023-09-29T23:29:39.638774+0000","last_became_peered":"2023-09-29T23:29:39.638774+0000","last_unstale":"2023-09-29T23:29:39.646545+0000","last_undegraded":"2023-09-29T23:29:39.646545+0000","last_fullsized":"2023-09-29T23:29:39.646545+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:37:30.614407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.636591+0000","last_change":"2023-09-29T23:29:39.633491+0000","last_active":"2023-09-29T23:29:39.636591+0000","last_peered":"2023-09-29T23:29:39.636591+0000","last_clean":"2023-09-29T23:29:39.636591+0000","last_became_active":"2023-09-29T23:29:39.633174+0000","last_became_peered":"2023-09-29T23:29:39.633174+0000","last_unstale":"2023-09-29T23:29:39.636591+0000","last_undegraded":"2023-09-29T23:29:39.636591+0000","last_fullsized":"2023-09-29T23:29:39.636591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:36:55.020338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625219+0000","last_change":"2023-09-29T23:29:39.613672+0000","last_active":"2023-09-29T23:29:39.625219+0000","last_peered":"2023-09-29T23:29:39.625219+0000","last_clean":"2023-09-29T23:29:39.625219+0000","last_became_active":"2023-09-29T23:29:39.612667+0000","last_became_peered":"2023-09-29T23:29:39.612667+0000","last_unstale":"2023-09-29T23:29:39.625219+0000","last_undegraded":"2023-09-29T23:29:39.625219+0000","last_fullsized":"2023-09-29T23:29:39.625219+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:47:46.021660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641070+0000","last_change":"2023-09-29T23:29:39.639206+0000","last_active":"2023-09-29T23:29:39.641070+0000","last_peered":"2023-09-29T23:29:39.641070+0000","last_clean":"2023-09-29T23:29:39.641070+0000","last_became_active":"2023-09-29T23:29:39.638930+0000","last_became_peered":"2023-09-29T23:29:39.638930+0000","last_unstale":"2023-09-29T23:29:39.641070+0000","last_undegraded":"2023-09-29T23:29:39.641070+0000","last_fullsized":"2023-09-29T23:29:39.641070+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:34:56.298163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629896+0000","last_change":"2023-09-29T23:29:39.621733+0000","last_active":"2023-09-29T23:29:39.629896+0000","last_peered":"2023-09-29T23:29:39.629896+0000","last_clean":"2023-09-29T23:29:39.629896+0000","last_became_active":"2023-09-29T23:29:39.621410+0000","last_became_peered":"2023-09-29T23:29:39.621410+0000","last_unstale":"2023-09-29T23:29:39.629896+0000","last_undegraded":"2023-09-29T23:29:39.629896+0000","last_fullsized":"2023-09-29T23:29:39.629896+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:13:11.017795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645339+0000","last_change":"2023-09-29T23:29:39.635888+0000","last_active":"2023-09-29T23:29:39.645339+0000","last_peered":"2023-09-29T23:29:39.645339+0000","last_clean":"2023-09-29T23:29:39.645339+0000","last_became_active":"2023-09-29T23:29:39.634120+0000","last_became_peered":"2023-09-29T23:29:39.634120+0000","last_unstale":"2023-09-29T23:29:39.645339+0000","last_undegraded":"2023-09-29T23:29:39.645339+0000","last_fullsized":"2023-09-29T23:29:39.645339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:16:01.339180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625208+0000","last_change":"2023-09-29T23:29:39.612568+0000","last_active":"2023-09-29T23:29:39.625208+0000","last_peered":"2023-09-29T23:29:39.625208+0000","last_clean":"2023-09-29T23:29:39.625208+0000","last_became_active":"2023-09-29T23:29:39.612349+0000","last_became_peered":"2023-09-29T23:29:39.612349+0000","last_unstale":"2023-09-29T23:29:39.625208+0000","last_undegraded":"2023-09-29T23:29:39.625208+0000","last_fullsized":"2023-09-29T23:29:39.625208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:36:55.020338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641527+0000","last_change":"2023-09-29T23:29:39.640244+0000","last_active":"2023-09-29T23:29:39.641527+0000","last_peered":"2023-09-29T23:29:39.641527+0000","last_clean":"2023-09-29T23:29:39.641527+0000","last_became_active":"2023-09-29T23:29:39.639928+0000","last_became_peered":"2023-09-29T23:29:39.639928+0000","last_unstale":"2023-09-29T23:29:39.641527+0000","last_undegraded":"2023-09-29T23:29:39.641527+0000","last_fullsized":"2023-09-29T23:29:39.641527+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:20:35.058663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.610083+0000","last_change":"2023-09-29T23:29:39.609392+0000","last_active":"2023-09-29T23:29:39.610083+0000","last_peered":"2023-09-29T23:29:39.610083+0000","last_clean":"2023-09-29T23:29:39.610083+0000","last_became_active":"2023-09-29T23:29:39.609137+0000","last_became_peered":"2023-09-29T23:29:39.609137+0000","last_unstale":"2023-09-29T23:29:39.610083+0000","last_undegraded":"2023-09-29T23:29:39.610083+0000","last_fullsized":"2023-09-29T23:29:39.610083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:14:21.471042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647538+0000","last_change":"2023-09-29T23:29:39.646351+0000","last_active":"2023-09-29T23:29:39.647538+0000","last_peered":"2023-09-29T23:29:39.647538+0000","last_clean":"2023-09-29T23:29:39.647538+0000","last_became_active":"2023-09-29T23:29:39.646091+0000","last_became_peered":"2023-09-29T23:29:39.646091+0000","last_unstale":"2023-09-29T23:29:39.647538+0000","last_undegraded":"2023-09-29T23:29:39.647538+0000","last_fullsized":"2023-09-29T23:29:39.647538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:41:52.519640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.606960+0000","last_change":"2023-09-29T23:29:39.606266+0000","last_active":"2023-09-29T23:29:39.606960+0000","last_peered":"2023-09-29T23:29:39.606960+0000","last_clean":"2023-09-29T23:29:39.606960+0000","last_became_active":"2023-09-29T23:29:39.606025+0000","last_became_peered":"2023-09-29T23:29:39.606025+0000","last_unstale":"2023-09-29T23:29:39.606960+0000","last_undegraded":"2023-09-29T23:29:39.606960+0000","last_fullsized":"2023-09-29T23:29:39.606960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T02:17:42.897947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637537+0000","last_change":"2023-09-29T23:29:39.634587+0000","last_active":"2023-09-29T23:29:39.637537+0000","last_peered":"2023-09-29T23:29:39.637537+0000","last_clean":"2023-09-29T23:29:39.637537+0000","last_became_active":"2023-09-29T23:29:39.634344+0000","last_became_peered":"2023-09-29T23:29:39.634344+0000","last_unstale":"2023-09-29T23:29:39.637537+0000","last_undegraded":"2023-09-29T23:29:39.637537+0000","last_fullsized":"2023-09-29T23:29:39.637537+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:20:37.281771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637839+0000","last_change":"2023-09-29T23:29:39.632991+0000","last_active":"2023-09-29T23:29:39.637839+0000","last_peered":"2023-09-29T23:29:39.637839+0000","last_clean":"2023-09-29T23:29:39.637839+0000","last_became_active":"2023-09-29T23:29:39.632721+0000","last_became_peered":"2023-09-29T23:29:39.632721+0000","last_unstale":"2023-09-29T23:29:39.637839+0000","last_undegraded":"2023-09-29T23:29:39.637839+0000","last_fullsized":"2023-09-29T23:29:39.637839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:25:55.111560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.618030+0000","last_change":"2023-09-29T23:29:39.616390+0000","last_active":"2023-09-29T23:29:39.618030+0000","last_peered":"2023-09-29T23:29:39.618030+0000","last_clean":"2023-09-29T23:29:39.618030+0000","last_became_active":"2023-09-29T23:29:39.615974+0000","last_became_peered":"2023-09-29T23:29:39.615974+0000","last_unstale":"2023-09-29T23:29:39.618030+0000","last_undegraded":"2023-09-29T23:29:39.618030+0000","last_fullsized":"2023-09-29T23:29:39.618030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:05:52.183625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.630420+0000","last_change":"2023-09-29T23:29:39.625067+0000","last_active":"2023-09-29T23:29:39.630420+0000","last_peered":"2023-09-29T23:29:39.630420+0000","last_clean":"2023-09-29T23:29:39.630420+0000","last_became_active":"2023-09-29T23:29:39.624793+0000","last_became_peered":"2023-09-29T23:29:39.624793+0000","last_unstale":"2023-09-29T23:29:39.630420+0000","last_undegraded":"2023-09-29T23:29:39.630420+0000","last_fullsized":"2023-09-29T23:29:39.630420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:48.157072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645155+0000","last_change":"2023-09-29T23:29:39.633681+0000","last_active":"2023-09-29T23:29:39.645155+0000","last_peered":"2023-09-29T23:29:39.645155+0000","last_clean":"2023-09-29T23:29:39.645155+0000","last_became_active":"2023-09-29T23:29:39.633400+0000","last_became_peered":"2023-09-29T23:29:39.633400+0000","last_unstale":"2023-09-29T23:29:39.645155+0000","last_undegraded":"2023-09-29T23:29:39.645155+0000","last_fullsized":"2023-09-29T23:29:39.645155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:29:56.190518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638194+0000","last_change":"2023-09-29T23:29:39.272166+0000","last_active":"2023-09-29T23:29:39.638194+0000","last_peered":"2023-09-29T23:29:39.638194+0000","last_clean":"2023-09-29T23:29:39.638194+0000","last_became_active":"2023-09-29T23:29:39.271616+0000","last_became_peered":"2023-09-29T23:29:39.271616+0000","last_unstale":"2023-09-29T23:29:39.638194+0000","last_undegraded":"2023-09-29T23:29:39.638194+0000","last_fullsized":"2023-09-29T23:29:39.638194+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:14:20.467285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647140+0000","last_change":"2023-09-29T23:29:39.639338+0000","last_active":"2023-09-29T23:29:39.647140+0000","last_peered":"2023-09-29T23:29:39.647140+0000","last_clean":"2023-09-29T23:29:39.647140+0000","last_became_active":"2023-09-29T23:29:39.639030+0000","last_became_peered":"2023-09-29T23:29:39.639030+0000","last_unstale":"2023-09-29T23:29:39.647140+0000","last_undegraded":"2023-09-29T23:29:39.647140+0000","last_fullsized":"2023-09-29T23:29:39.647140+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:46:49.801617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.598222+0000","last_change":"2023-09-29T23:29:39.269808+0000","last_active":"2023-09-29T23:29:39.598222+0000","last_peered":"2023-09-29T23:29:39.598222+0000","last_clean":"2023-09-29T23:29:39.598222+0000","last_became_active":"2023-09-29T23:29:39.269509+0000","last_became_peered":"2023-09-29T23:29:39.269509+0000","last_unstale":"2023-09-29T23:29:39.598222+0000","last_undegraded":"2023-09-29T23:29:39.598222+0000","last_fullsized":"2023-09-29T23:29:39.598222+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:47:31.314319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.646474+0000","last_change":"2023-09-29T23:29:39.639099+0000","last_active":"2023-09-29T23:29:39.646474+0000","last_peered":"2023-09-29T23:29:39.646474+0000","last_clean":"2023-09-29T23:29:39.646474+0000","last_became_active":"2023-09-29T23:29:39.636608+0000","last_became_peered":"2023-09-29T23:29:39.636608+0000","last_unstale":"2023-09-29T23:29:39.646474+0000","last_undegraded":"2023-09-29T23:29:39.646474+0000","last_fullsized":"2023-09-29T23:29:39.646474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:24:20.819335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620024+0000","last_change":"2023-09-29T23:29:39.273761+0000","last_active":"2023-09-29T23:29:39.620024+0000","last_peered":"2023-09-29T23:29:39.620024+0000","last_clean":"2023-09-29T23:29:39.620024+0000","last_became_active":"2023-09-29T23:29:39.273376+0000","last_became_peered":"2023-09-29T23:29:39.273376+0000","last_unstale":"2023-09-29T23:29:39.620024+0000","last_undegraded":"2023-09-29T23:29:39.620024+0000","last_fullsized":"2023-09-29T23:29:39.620024+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:59:21.890466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.636454+0000","last_change":"2023-09-29T23:29:39.631948+0000","last_active":"2023-09-29T23:29:39.636454+0000","last_peered":"2023-09-29T23:29:39.636454+0000","last_clean":"2023-09-29T23:29:39.636454+0000","last_became_active":"2023-09-29T23:29:39.631531+0000","last_became_peered":"2023-09-29T23:29:39.631531+0000","last_unstale":"2023-09-29T23:29:39.636454+0000","last_undegraded":"2023-09-29T23:29:39.636454+0000","last_fullsized":"2023-09-29T23:29:39.636454+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:04:09.469416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"18'8","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645828+0000","last_change":"2023-09-29T23:29:39.259965+0000","last_active":"2023-09-29T23:29:39.645828+0000","last_peered":"2023-09-29T23:29:39.645828+0000","last_clean":"2023-09-29T23:29:39.645828+0000","last_became_active":"2023-09-29T23:29:39.259617+0000","last_became_peered":"2023-09-29T23:29:39.259617+0000","last_unstale":"2023-09-29T23:29:39.645828+0000","last_undegraded":"2023-09-29T23:29:39.645828+0000","last_fullsized":"2023-09-29T23:29:39.645828+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:43:32.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.f","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.630444+0000","last_change":"2023-09-29T23:29:39.262801+0000","last_active":"2023-09-29T23:29:39.630444+0000","last_peered":"2023-09-29T23:29:39.630444+0000","last_clean":"2023-09-29T23:29:39.630444+0000","last_became_active":"2023-09-29T23:29:39.259692+0000","last_became_peered":"2023-09-29T23:29:39.259692+0000","last_unstale":"2023-09-29T23:29:39.630444+0000","last_undegraded":"2023-09-29T23:29:39.630444+0000","last_fullsized":"2023-09-29T23:29:39.630444+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:43:32.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645777+0000","last_change":"2023-09-29T23:29:39.636515+0000","last_active":"2023-09-29T23:29:39.645777+0000","last_peered":"2023-09-29T23:29:39.645777+0000","last_clean":"2023-09-29T23:29:39.645777+0000","last_became_active":"2023-09-29T23:29:39.633532+0000","last_became_peered":"2023-09-29T23:29:39.633532+0000","last_unstale":"2023-09-29T23:29:39.645777+0000","last_undegraded":"2023-09-29T23:29:39.645777+0000","last_fullsized":"2023-09-29T23:29:39.645777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:53:43.483894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625041+0000","last_change":"2023-09-29T23:29:39.268225+0000","last_active":"2023-09-29T23:29:39.625041+0000","last_peered":"2023-09-29T23:29:39.625041+0000","last_clean":"2023-09-29T23:29:39.625041+0000","last_became_active":"2023-09-29T23:29:39.267910+0000","last_became_peered":"2023-09-29T23:29:39.267910+0000","last_unstale":"2023-09-29T23:29:39.625041+0000","last_undegraded":"2023-09-29T23:29:39.625041+0000","last_fullsized":"2023-09-29T23:29:39.625041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:58:28.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.624786+0000","last_change":"2023-09-29T23:29:39.612447+0000","last_active":"2023-09-29T23:29:39.624786+0000","last_peered":"2023-09-29T23:29:39.624786+0000","last_clean":"2023-09-29T23:29:39.624786+0000","last_became_active":"2023-09-29T23:29:39.611905+0000","last_became_peered":"2023-09-29T23:29:39.611905+0000","last_unstale":"2023-09-29T23:29:39.624786+0000","last_undegraded":"2023-09-29T23:29:39.624786+0000","last_fullsized":"2023-09-29T23:29:39.624786+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:31.379677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645233+0000","last_change":"2023-09-29T23:29:39.270818+0000","last_active":"2023-09-29T23:29:39.645233+0000","last_peered":"2023-09-29T23:29:39.645233+0000","last_clean":"2023-09-29T23:29:39.645233+0000","last_became_active":"2023-09-29T23:29:39.270419+0000","last_became_peered":"2023-09-29T23:29:39.270419+0000","last_unstale":"2023-09-29T23:29:39.645233+0000","last_undegraded":"2023-09-29T23:29:39.645233+0000","last_fullsized":"2023-09-29T23:29:39.645233+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:39:03.054065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638105+0000","last_change":"2023-09-29T23:29:39.634163+0000","last_active":"2023-09-29T23:29:39.638105+0000","last_peered":"2023-09-29T23:29:39.638105+0000","last_clean":"2023-09-29T23:29:39.638105+0000","last_became_active":"2023-09-29T23:29:39.633902+0000","last_became_peered":"2023-09-29T23:29:39.633902+0000","last_unstale":"2023-09-29T23:29:39.638105+0000","last_undegraded":"2023-09-29T23:29:39.638105+0000","last_fullsized":"2023-09-29T23:29:39.638105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:48.157072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.616087+0000","last_change":"2023-09-29T23:29:39.267422+0000","last_active":"2023-09-29T23:29:39.616087+0000","last_peered":"2023-09-29T23:29:39.616087+0000","last_clean":"2023-09-29T23:29:39.616087+0000","last_became_active":"2023-09-29T23:29:39.267119+0000","last_became_peered":"2023-09-29T23:29:39.267119+0000","last_unstale":"2023-09-29T23:29:39.616087+0000","last_undegraded":"2023-09-29T23:29:39.616087+0000","last_fullsized":"2023-09-29T23:29:39.616087+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:29:06.658217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609399+0000","last_change":"2023-09-29T23:29:39.607523+0000","last_active":"2023-09-29T23:29:39.609399+0000","last_peered":"2023-09-29T23:29:39.609399+0000","last_clean":"2023-09-29T23:29:39.609399+0000","last_became_active":"2023-09-29T23:29:39.607172+0000","last_became_peered":"2023-09-29T23:29:39.607172+0000","last_unstale":"2023-09-29T23:29:39.609399+0000","last_undegraded":"2023-09-29T23:29:39.609399+0000","last_fullsized":"2023-09-29T23:29:39.609399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:05:52.183625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.614202+0000","last_change":"2023-09-29T23:29:39.259718+0000","last_active":"2023-09-29T23:29:39.614202+0000","last_peered":"2023-09-29T23:29:39.614202+0000","last_clean":"2023-09-29T23:29:39.614202+0000","last_became_active":"2023-09-29T23:29:39.259429+0000","last_became_peered":"2023-09-29T23:29:39.259429+0000","last_unstale":"2023-09-29T23:29:39.614202+0000","last_undegraded":"2023-09-29T23:29:39.614202+0000","last_fullsized":"2023-09-29T23:29:39.614202+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:18:38.314574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.635728+0000","last_change":"2023-09-29T23:29:39.631644+0000","last_active":"2023-09-29T23:29:39.635728+0000","last_peered":"2023-09-29T23:29:39.635728+0000","last_clean":"2023-09-29T23:29:39.635728+0000","last_became_active":"2023-09-29T23:29:39.630542+0000","last_became_peered":"2023-09-29T23:29:39.630542+0000","last_unstale":"2023-09-29T23:29:39.635728+0000","last_undegraded":"2023-09-29T23:29:39.635728+0000","last_fullsized":"2023-09-29T23:29:39.635728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:40:30.851104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.613760+0000","last_change":"2023-09-29T23:29:39.268889+0000","last_active":"2023-09-29T23:29:39.613760+0000","last_peered":"2023-09-29T23:29:39.613760+0000","last_clean":"2023-09-29T23:29:39.613760+0000","last_became_active":"2023-09-29T23:29:39.268443+0000","last_became_peered":"2023-09-29T23:29:39.268443+0000","last_unstale":"2023-09-29T23:29:39.613760+0000","last_undegraded":"2023-09-29T23:29:39.613760+0000","last_fullsized":"2023-09-29T23:29:39.613760+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:58:34.772816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647157+0000","last_change":"2023-09-29T23:29:39.639561+0000","last_active":"2023-09-29T23:29:39.647157+0000","last_peered":"2023-09-29T23:29:39.647157+0000","last_clean":"2023-09-29T23:29:39.647157+0000","last_became_active":"2023-09-29T23:29:39.639082+0000","last_became_peered":"2023-09-29T23:29:39.639082+0000","last_unstale":"2023-09-29T23:29:39.647157+0000","last_undegraded":"2023-09-29T23:29:39.647157+0000","last_fullsized":"2023-09-29T23:29:39.647157+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:15:31.512106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620049+0000","last_change":"2023-09-29T23:29:39.274677+0000","last_active":"2023-09-29T23:29:39.620049+0000","last_peered":"2023-09-29T23:29:39.620049+0000","last_clean":"2023-09-29T23:29:39.620049+0000","last_became_active":"2023-09-29T23:29:39.274395+0000","last_became_peered":"2023-09-29T23:29:39.274395+0000","last_unstale":"2023-09-29T23:29:39.620049+0000","last_undegraded":"2023-09-29T23:29:39.620049+0000","last_fullsized":"2023-09-29T23:29:39.620049+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:48.655823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.638074+0000","last_change":"2023-09-29T23:29:39.633620+0000","last_active":"2023-09-29T23:29:39.638074+0000","last_peered":"2023-09-29T23:29:39.638074+0000","last_clean":"2023-09-29T23:29:39.638074+0000","last_became_active":"2023-09-29T23:29:39.633346+0000","last_became_peered":"2023-09-29T23:29:39.633346+0000","last_unstale":"2023-09-29T23:29:39.638074+0000","last_undegraded":"2023-09-29T23:29:39.638074+0000","last_fullsized":"2023-09-29T23:29:39.638074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:59:22.894223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617510+0000","last_change":"2023-09-29T23:29:39.269804+0000","last_active":"2023-09-29T23:29:39.617510+0000","last_peered":"2023-09-29T23:29:39.617510+0000","last_clean":"2023-09-29T23:29:39.617510+0000","last_became_active":"2023-09-29T23:29:39.269530+0000","last_became_peered":"2023-09-29T23:29:39.269530+0000","last_unstale":"2023-09-29T23:29:39.617510+0000","last_undegraded":"2023-09-29T23:29:39.617510+0000","last_fullsized":"2023-09-29T23:29:39.617510+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:47:31.314319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647052+0000","last_change":"2023-09-29T23:29:39.645616+0000","last_active":"2023-09-29T23:29:39.647052+0000","last_peered":"2023-09-29T23:29:39.647052+0000","last_clean":"2023-09-29T23:29:39.647052+0000","last_became_active":"2023-09-29T23:29:39.644961+0000","last_became_peered":"2023-09-29T23:29:39.644961+0000","last_unstale":"2023-09-29T23:29:39.647052+0000","last_undegraded":"2023-09-29T23:29:39.647052+0000","last_fullsized":"2023-09-29T23:29:39.647052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:56:40.493860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'117","reported_seq":166,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.612078+0000","last_change":"2023-09-29T23:29:29.623917+0000","last_active":"2023-09-29T23:29:39.612078+0000","last_peered":"2023-09-29T23:29:39.612078+0000","last_clean":"2023-09-29T23:29:39.612078+0000","last_became_active":"2023-09-29T23:29:29.621470+0000","last_became_peered":"2023-09-29T23:29:29.621470+0000","last_unstale":"2023-09-29T23:29:39.612078+0000","last_undegraded":"2023-09-29T23:29:39.612078+0000","last_fullsized":"2023-09-29T23:29:39.612078+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:28.522101+0000","last_clean_scrub_stamp":"2023-09-29T23:29:28.522101+0000","objects_scrubbed":0,"log_size":117,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:37:09.181891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620013+0000","last_change":"2023-09-29T23:29:35.099334+0000","last_active":"2023-09-29T23:29:39.620013+0000","last_peered":"2023-09-29T23:29:39.620013+0000","last_clean":"2023-09-29T23:29:39.620013+0000","last_became_active":"2023-09-29T23:29:35.098819+0000","last_became_peered":"2023-09-29T23:29:35.098819+0000","last_unstale":"2023-09-29T23:29:39.620013+0000","last_undegraded":"2023-09-29T23:29:39.620013+0000","last_fullsized":"2023-09-29T23:29:39.620013+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:29:21.699082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617326+0000","last_change":"2023-09-29T23:29:39.272150+0000","last_active":"2023-09-29T23:29:39.617326+0000","last_peered":"2023-09-29T23:29:39.617326+0000","last_clean":"2023-09-29T23:29:39.617326+0000","last_became_active":"2023-09-29T23:29:39.271819+0000","last_became_peered":"2023-09-29T23:29:39.271819+0000","last_unstale":"2023-09-29T23:29:39.617326+0000","last_undegraded":"2023-09-29T23:29:39.617326+0000","last_fullsized":"2023-09-29T23:29:39.617326+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:56:47.153315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647893+0000","last_change":"2023-09-29T23:29:39.646963+0000","last_active":"2023-09-29T23:29:39.647893+0000","last_peered":"2023-09-29T23:29:39.647893+0000","last_clean":"2023-09-29T23:29:39.647893+0000","last_became_active":"2023-09-29T23:29:39.646666+0000","last_became_peered":"2023-09-29T23:29:39.646666+0000","last_unstale":"2023-09-29T23:29:39.647893+0000","last_undegraded":"2023-09-29T23:29:39.647893+0000","last_fullsized":"2023-09-29T23:29:39.647893+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:57:10.275706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617601+0000","last_change":"2023-09-29T23:29:35.098951+0000","last_active":"2023-09-29T23:29:39.617601+0000","last_peered":"2023-09-29T23:29:39.617601+0000","last_clean":"2023-09-29T23:29:39.617601+0000","last_became_active":"2023-09-29T23:29:35.098555+0000","last_became_peered":"2023-09-29T23:29:35.098555+0000","last_unstale":"2023-09-29T23:29:39.617601+0000","last_undegraded":"2023-09-29T23:29:39.617601+0000","last_fullsized":"2023-09-29T23:29:39.617601+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-09-30T23:43:14.246995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597396+0000","last_change":"2023-09-29T23:29:39.260929+0000","last_active":"2023-09-29T23:29:39.597396+0000","last_peered":"2023-09-29T23:29:39.597396+0000","last_clean":"2023-09-29T23:29:39.597396+0000","last_became_active":"2023-09-29T23:29:39.260563+0000","last_became_peered":"2023-09-29T23:29:39.260563+0000","last_unstale":"2023-09-29T23:29:39.597396+0000","last_undegraded":"2023-09-29T23:29:39.597396+0000","last_fullsized":"2023-09-29T23:29:39.597396+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:09:46.708805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.646512+0000","last_change":"2023-09-29T23:29:39.639295+0000","last_active":"2023-09-29T23:29:39.646512+0000","last_peered":"2023-09-29T23:29:39.646512+0000","last_clean":"2023-09-29T23:29:39.646512+0000","last_became_active":"2023-09-29T23:29:39.638232+0000","last_became_peered":"2023-09-29T23:29:39.638232+0000","last_unstale":"2023-09-29T23:29:39.646512+0000","last_undegraded":"2023-09-29T23:29:39.646512+0000","last_fullsized":"2023-09-29T23:29:39.646512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:45:45.006295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.598274+0000","last_change":"2023-09-29T23:29:35.095306+0000","last_active":"2023-09-29T23:29:39.598274+0000","last_peered":"2023-09-29T23:29:39.598274+0000","last_clean":"2023-09-29T23:29:39.598274+0000","last_became_active":"2023-09-29T23:29:35.093309+0000","last_became_peered":"2023-09-29T23:29:35.093309+0000","last_unstale":"2023-09-29T23:29:39.598274+0000","last_undegraded":"2023-09-29T23:29:39.598274+0000","last_fullsized":"2023-09-29T23:29:39.598274+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:10:21.305466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.595615+0000","last_change":"2023-09-29T23:29:39.261319+0000","last_active":"2023-09-29T23:29:39.595615+0000","last_peered":"2023-09-29T23:29:39.595615+0000","last_clean":"2023-09-29T23:29:39.595615+0000","last_became_active":"2023-09-29T23:29:39.260979+0000","last_became_peered":"2023-09-29T23:29:39.260979+0000","last_unstale":"2023-09-29T23:29:39.595615+0000","last_undegraded":"2023-09-29T23:29:39.595615+0000","last_fullsized":"2023-09-29T23:29:39.595615+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:58:28.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.625251+0000","last_change":"2023-09-29T23:29:39.620458+0000","last_active":"2023-09-29T23:29:39.625251+0000","last_peered":"2023-09-29T23:29:39.625251+0000","last_clean":"2023-09-29T23:29:39.625251+0000","last_became_active":"2023-09-29T23:29:39.620060+0000","last_became_peered":"2023-09-29T23:29:39.620060+0000","last_unstale":"2023-09-29T23:29:39.625251+0000","last_undegraded":"2023-09-29T23:29:39.625251+0000","last_fullsized":"2023-09-29T23:29:39.625251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:49.659581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617399+0000","last_change":"2023-09-29T23:29:35.099643+0000","last_active":"2023-09-29T23:29:39.617399+0000","last_peered":"2023-09-29T23:29:39.617399+0000","last_clean":"2023-09-29T23:29:39.617399+0000","last_became_active":"2023-09-29T23:29:35.099061+0000","last_became_peered":"2023-09-29T23:29:35.099061+0000","last_unstale":"2023-09-29T23:29:39.617399+0000","last_undegraded":"2023-09-29T23:29:39.617399+0000","last_fullsized":"2023-09-29T23:29:39.617399+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:30.050339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.599401+0000","last_change":"2023-09-29T23:29:39.259645+0000","last_active":"2023-09-29T23:29:39.599401+0000","last_peered":"2023-09-29T23:29:39.599401+0000","last_clean":"2023-09-29T23:29:39.599401+0000","last_became_active":"2023-09-29T23:29:39.258895+0000","last_became_peered":"2023-09-29T23:29:39.258895+0000","last_unstale":"2023-09-29T23:29:39.599401+0000","last_undegraded":"2023-09-29T23:29:39.599401+0000","last_fullsized":"2023-09-29T23:29:39.599401+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:32:59.650256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609649+0000","last_change":"2023-09-29T23:29:39.608553+0000","last_active":"2023-09-29T23:29:39.609649+0000","last_peered":"2023-09-29T23:29:39.609649+0000","last_clean":"2023-09-29T23:29:39.609649+0000","last_became_active":"2023-09-29T23:29:39.608249+0000","last_became_peered":"2023-09-29T23:29:39.608249+0000","last_unstale":"2023-09-29T23:29:39.609649+0000","last_undegraded":"2023-09-29T23:29:39.609649+0000","last_fullsized":"2023-09-29T23:29:39.609649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:38:28.534318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.599877+0000","last_change":"2023-09-29T23:29:39.259646+0000","last_active":"2023-09-29T23:29:39.599877+0000","last_peered":"2023-09-29T23:29:39.599877+0000","last_clean":"2023-09-29T23:29:39.599877+0000","last_became_active":"2023-09-29T23:29:39.259285+0000","last_became_peered":"2023-09-29T23:29:39.259285+0000","last_unstale":"2023-09-29T23:29:39.599877+0000","last_undegraded":"2023-09-29T23:29:39.599877+0000","last_fullsized":"2023-09-29T23:29:39.599877+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:47:29.475878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597541+0000","last_change":"2023-09-29T23:29:35.100113+0000","last_active":"2023-09-29T23:29:39.597541+0000","last_peered":"2023-09-29T23:29:39.597541+0000","last_clean":"2023-09-29T23:29:39.597541+0000","last_became_active":"2023-09-29T23:29:35.099578+0000","last_became_peered":"2023-09-29T23:29:35.099578+0000","last_unstale":"2023-09-29T23:29:39.597541+0000","last_undegraded":"2023-09-29T23:29:39.597541+0000","last_fullsized":"2023-09-29T23:29:39.597541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:57:39.508565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.640050+0000","last_change":"2023-09-29T23:29:39.637174+0000","last_active":"2023-09-29T23:29:39.640050+0000","last_peered":"2023-09-29T23:29:39.640050+0000","last_clean":"2023-09-29T23:29:39.640050+0000","last_became_active":"2023-09-29T23:29:39.636903+0000","last_became_peered":"2023-09-29T23:29:39.636903+0000","last_unstale":"2023-09-29T23:29:39.640050+0000","last_undegraded":"2023-09-29T23:29:39.640050+0000","last_fullsized":"2023-09-29T23:29:39.640050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:04:25.929196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597410+0000","last_change":"2023-09-29T23:29:39.271080+0000","last_active":"2023-09-29T23:29:39.597410+0000","last_peered":"2023-09-29T23:29:39.597410+0000","last_clean":"2023-09-29T23:29:39.597410+0000","last_became_active":"2023-09-29T23:29:39.270673+0000","last_became_peered":"2023-09-29T23:29:39.270673+0000","last_unstale":"2023-09-29T23:29:39.597410+0000","last_undegraded":"2023-09-29T23:29:39.597410+0000","last_fullsized":"2023-09-29T23:29:39.597410+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:28:04.709351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.628891+0000","last_change":"2023-09-29T23:29:35.099325+0000","last_active":"2023-09-29T23:29:39.628891+0000","last_peered":"2023-09-29T23:29:39.628891+0000","last_clean":"2023-09-29T23:29:39.628891+0000","last_became_active":"2023-09-29T23:29:35.098701+0000","last_became_peered":"2023-09-29T23:29:35.098701+0000","last_unstale":"2023-09-29T23:29:39.628891+0000","last_undegraded":"2023-09-29T23:29:39.628891+0000","last_fullsized":"2023-09-29T23:29:39.628891+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:35:07.769754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.641024+0000","last_change":"2023-09-29T23:29:39.638042+0000","last_active":"2023-09-29T23:29:39.641024+0000","last_peered":"2023-09-29T23:29:39.641024+0000","last_clean":"2023-09-29T23:29:39.641024+0000","last_became_active":"2023-09-29T23:29:39.637714+0000","last_became_peered":"2023-09-29T23:29:39.637714+0000","last_unstale":"2023-09-29T23:29:39.641024+0000","last_undegraded":"2023-09-29T23:29:39.641024+0000","last_fullsized":"2023-09-29T23:29:39.641024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:47:58.354132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.644775+0000","last_change":"2023-09-29T23:29:39.269267+0000","last_active":"2023-09-29T23:29:39.644775+0000","last_peered":"2023-09-29T23:29:39.644775+0000","last_clean":"2023-09-29T23:29:39.644775+0000","last_became_active":"2023-09-29T23:29:39.268949+0000","last_became_peered":"2023-09-29T23:29:39.268949+0000","last_unstale":"2023-09-29T23:29:39.644775+0000","last_undegraded":"2023-09-29T23:29:39.644775+0000","last_fullsized":"2023-09-29T23:29:39.644775+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:07:08.485719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.597483+0000","last_change":"2023-09-29T23:29:35.096994+0000","last_active":"2023-09-29T23:29:39.597483+0000","last_peered":"2023-09-29T23:29:39.597483+0000","last_clean":"2023-09-29T23:29:39.597483+0000","last_became_active":"2023-09-29T23:29:35.096568+0000","last_became_peered":"2023-09-29T23:29:35.096568+0000","last_unstale":"2023-09-29T23:29:39.597483+0000","last_undegraded":"2023-09-29T23:29:39.597483+0000","last_fullsized":"2023-09-29T23:29:39.597483+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:22:06.698301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645797+0000","last_change":"2023-09-29T23:29:39.638385+0000","last_active":"2023-09-29T23:29:39.645797+0000","last_peered":"2023-09-29T23:29:39.645797+0000","last_clean":"2023-09-29T23:29:39.645797+0000","last_became_active":"2023-09-29T23:29:39.634957+0000","last_became_peered":"2023-09-29T23:29:39.634957+0000","last_unstale":"2023-09-29T23:29:39.645797+0000","last_undegraded":"2023-09-29T23:29:39.645797+0000","last_fullsized":"2023-09-29T23:29:39.645797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:13:11.017795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-29T23:29:41.577250+0000","last_change":"2023-09-29T23:29:39.271004+0000","last_active":"2023-09-29T23:29:41.577250+0000","last_peered":"2023-09-29T23:29:41.577250+0000","last_clean":"2023-09-29T23:29:41.577250+0000","last_became_active":"2023-09-29T23:29:39.270721+0000","last_became_peered":"2023-09-29T23:29:39.270721+0000","last_unstale":"2023-09-29T23:29:41.577250+0000","last_undegraded":"2023-09-29T23:29:41.577250+0000","last_fullsized":"2023-09-29T23:29:41.577250+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:18:38.314574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.620608+0000","last_change":"2023-09-29T23:29:35.776424+0000","last_active":"2023-09-29T23:29:39.620608+0000","last_peered":"2023-09-29T23:29:39.620608+0000","last_clean":"2023-09-29T23:29:39.620608+0000","last_became_active":"2023-09-29T23:29:35.775625+0000","last_became_peered":"2023-09-29T23:29:35.775625+0000","last_unstale":"2023-09-29T23:29:39.620608+0000","last_undegraded":"2023-09-29T23:29:39.620608+0000","last_fullsized":"2023-09-29T23:29:39.620608+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:34.076891+0000","last_clean_scrub_stamp":"2023-09-29T23:29:34.076891+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:46.078517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.627165+0000","last_change":"2023-09-29T23:29:39.624813+0000","last_active":"2023-09-29T23:29:39.627165+0000","last_peered":"2023-09-29T23:29:39.627165+0000","last_clean":"2023-09-29T23:29:39.627165+0000","last_became_active":"2023-09-29T23:29:39.624521+0000","last_became_peered":"2023-09-29T23:29:39.624521+0000","last_unstale":"2023-09-29T23:29:39.627165+0000","last_undegraded":"2023-09-29T23:29:39.627165+0000","last_fullsized":"2023-09-29T23:29:39.627165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:38:28.534318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.639914+0000","last_change":"2023-09-29T23:29:39.635422+0000","last_active":"2023-09-29T23:29:39.639914+0000","last_peered":"2023-09-29T23:29:39.639914+0000","last_clean":"2023-09-29T23:29:39.639914+0000","last_became_active":"2023-09-29T23:29:39.635139+0000","last_became_peered":"2023-09-29T23:29:39.635139+0000","last_unstale":"2023-09-29T23:29:39.639914+0000","last_undegraded":"2023-09-29T23:29:39.639914+0000","last_fullsized":"2023-09-29T23:29:39.639914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:04:09.469416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"18'6","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647558+0000","last_change":"2023-09-29T23:29:39.271958+0000","last_active":"2023-09-29T23:29:39.647558+0000","last_peered":"2023-09-29T23:29:39.647558+0000","last_clean":"2023-09-29T23:29:39.647558+0000","last_became_active":"2023-09-29T23:29:39.271601+0000","last_became_peered":"2023-09-29T23:29:39.271601+0000","last_unstale":"2023-09-29T23:29:39.647558+0000","last_undegraded":"2023-09-29T23:29:39.647558+0000","last_fullsized":"2023-09-29T23:29:39.647558+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:20:36.278014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637865+0000","last_change":"2023-09-29T23:29:39.633505+0000","last_active":"2023-09-29T23:29:39.637865+0000","last_peered":"2023-09-29T23:29:39.637865+0000","last_clean":"2023-09-29T23:29:39.637865+0000","last_became_active":"2023-09-29T23:29:39.633253+0000","last_became_peered":"2023-09-29T23:29:39.633253+0000","last_unstale":"2023-09-29T23:29:39.637865+0000","last_undegraded":"2023-09-29T23:29:39.637865+0000","last_fullsized":"2023-09-29T23:29:39.637865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:53:25.888059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"18'3","reported_seq":13,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.633016+0000","last_change":"2023-09-29T23:29:39.268518+0000","last_active":"2023-09-29T23:29:39.633016+0000","last_peered":"2023-09-29T23:29:39.633016+0000","last_clean":"2023-09-29T23:29:39.633016+0000","last_became_active":"2023-09-29T23:29:39.267411+0000","last_became_peered":"2023-09-29T23:29:39.267411+0000","last_unstale":"2023-09-29T23:29:39.633016+0000","last_undegraded":"2023-09-29T23:29:39.633016+0000","last_fullsized":"2023-09-29T23:29:39.633016+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:18:38.314574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.601527+0000","last_change":"2023-09-29T23:29:39.267677+0000","last_active":"2023-09-29T23:29:39.601527+0000","last_peered":"2023-09-29T23:29:39.601527+0000","last_clean":"2023-09-29T23:29:39.601527+0000","last_became_active":"2023-09-29T23:29:39.267374+0000","last_became_peered":"2023-09-29T23:29:39.267374+0000","last_unstale":"2023-09-29T23:29:39.601527+0000","last_undegraded":"2023-09-29T23:29:39.601527+0000","last_fullsized":"2023-09-29T23:29:39.601527+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:23:58.356932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.637457+0000","last_change":"2023-09-29T23:29:39.633662+0000","last_active":"2023-09-29T23:29:39.637457+0000","last_peered":"2023-09-29T23:29:39.637457+0000","last_clean":"2023-09-29T23:29:39.637457+0000","last_became_active":"2023-09-29T23:29:39.633327+0000","last_became_peered":"2023-09-29T23:29:39.633327+0000","last_unstale":"2023-09-29T23:29:39.637457+0000","last_undegraded":"2023-09-29T23:29:39.637457+0000","last_fullsized":"2023-09-29T23:29:39.637457+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T09:59:22.894223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.618457+0000","last_change":"2023-09-29T23:29:39.260818+0000","last_active":"2023-09-29T23:29:39.618457+0000","last_peered":"2023-09-29T23:29:39.618457+0000","last_clean":"2023-09-29T23:29:39.618457+0000","last_became_active":"2023-09-29T23:29:39.260529+0000","last_became_peered":"2023-09-29T23:29:39.260529+0000","last_unstale":"2023-09-29T23:29:39.618457+0000","last_undegraded":"2023-09-29T23:29:39.618457+0000","last_fullsized":"2023-09-29T23:29:39.618457+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T07:47:31.314319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645754+0000","last_change":"2023-09-29T23:29:39.635312+0000","last_active":"2023-09-29T23:29:39.645754+0000","last_peered":"2023-09-29T23:29:39.645754+0000","last_clean":"2023-09-29T23:29:39.645754+0000","last_became_active":"2023-09-29T23:29:39.633112+0000","last_became_peered":"2023-09-29T23:29:39.633112+0000","last_unstale":"2023-09-29T23:29:39.645754+0000","last_undegraded":"2023-09-29T23:29:39.645754+0000","last_fullsized":"2023-09-29T23:29:39.645754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:55:52.762563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645311+0000","last_change":"2023-09-29T23:29:39.635051+0000","last_active":"2023-09-29T23:29:39.645311+0000","last_peered":"2023-09-29T23:29:39.645311+0000","last_clean":"2023-09-29T23:29:39.645311+0000","last_became_active":"2023-09-29T23:29:39.632920+0000","last_became_peered":"2023-09-29T23:29:39.632920+0000","last_unstale":"2023-09-29T23:29:39.645311+0000","last_undegraded":"2023-09-29T23:29:39.645311+0000","last_fullsized":"2023-09-29T23:29:39.645311+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T10:55:37.158884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617319+0000","last_change":"2023-09-29T23:29:39.268725+0000","last_active":"2023-09-29T23:29:39.617319+0000","last_peered":"2023-09-29T23:29:39.617319+0000","last_clean":"2023-09-29T23:29:39.617319+0000","last_became_active":"2023-09-29T23:29:39.268360+0000","last_became_peered":"2023-09-29T23:29:39.268360+0000","last_unstale":"2023-09-29T23:29:39.617319+0000","last_undegraded":"2023-09-29T23:29:39.617319+0000","last_fullsized":"2023-09-29T23:29:39.617319+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:23:58.356932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.617990+0000","last_change":"2023-09-29T23:29:39.616291+0000","last_active":"2023-09-29T23:29:39.617990+0000","last_peered":"2023-09-29T23:29:39.617990+0000","last_clean":"2023-09-29T23:29:39.617990+0000","last_became_active":"2023-09-29T23:29:39.615966+0000","last_became_peered":"2023-09-29T23:29:39.615966+0000","last_unstale":"2023-09-29T23:29:39.617990+0000","last_undegraded":"2023-09-29T23:29:39.617990+0000","last_fullsized":"2023-09-29T23:29:39.617990+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:07:09.489477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.605702+0000","last_change":"2023-09-29T23:29:39.259196+0000","last_active":"2023-09-29T23:29:39.605702+0000","last_peered":"2023-09-29T23:29:39.605702+0000","last_clean":"2023-09-29T23:29:39.605702+0000","last_became_active":"2023-09-29T23:29:39.258785+0000","last_became_peered":"2023-09-29T23:29:39.258785+0000","last_unstale":"2023-09-29T23:29:39.605702+0000","last_undegraded":"2023-09-29T23:29:39.605702+0000","last_fullsized":"2023-09-29T23:29:39.605702+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:23:58.356932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.647103+0000","last_change":"2023-09-29T23:29:39.635946+0000","last_active":"2023-09-29T23:29:39.647103+0000","last_peered":"2023-09-29T23:29:39.647103+0000","last_clean":"2023-09-29T23:29:39.647103+0000","last_became_active":"2023-09-29T23:29:39.634129+0000","last_became_peered":"2023-09-29T23:29:39.634129+0000","last_unstale":"2023-09-29T23:29:39.647103+0000","last_undegraded":"2023-09-29T23:29:39.647103+0000","last_fullsized":"2023-09-29T23:29:39.647103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:31:36.612565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.629612+0000","last_change":"2023-09-29T23:29:39.252290+0000","last_active":"2023-09-29T23:29:39.629612+0000","last_peered":"2023-09-29T23:29:39.629612+0000","last_clean":"2023-09-29T23:29:39.629612+0000","last_became_active":"2023-09-29T23:29:39.246835+0000","last_became_peered":"2023-09-29T23:29:39.246835+0000","last_unstale":"2023-09-29T23:29:39.629612+0000","last_undegraded":"2023-09-29T23:29:39.629612+0000","last_fullsized":"2023-09-29T23:29:39.629612+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:09:46.708805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609639+0000","last_change":"2023-09-29T23:29:39.608519+0000","last_active":"2023-09-29T23:29:39.609639+0000","last_peered":"2023-09-29T23:29:39.609639+0000","last_clean":"2023-09-29T23:29:39.609639+0000","last_became_active":"2023-09-29T23:29:39.608334+0000","last_became_peered":"2023-09-29T23:29:39.608334+0000","last_unstale":"2023-09-29T23:29:39.609639+0000","last_undegraded":"2023-09-29T23:29:39.609639+0000","last_fullsized":"2023-09-29T23:29:39.609639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T01:39:31.379677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.632767+0000","last_change":"2023-09-29T23:29:39.270477+0000","last_active":"2023-09-29T23:29:39.632767+0000","last_peered":"2023-09-29T23:29:39.632767+0000","last_clean":"2023-09-29T23:29:39.632767+0000","last_became_active":"2023-09-29T23:29:39.270178+0000","last_became_peered":"2023-09-29T23:29:39.270178+0000","last_unstale":"2023-09-29T23:29:39.632767+0000","last_undegraded":"2023-09-29T23:29:39.632767+0000","last_fullsized":"2023-09-29T23:29:39.632767+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:05:51.179867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.608417+0000","last_change":"2023-09-29T23:29:39.605080+0000","last_active":"2023-09-29T23:29:39.608417+0000","last_peered":"2023-09-29T23:29:39.608417+0000","last_clean":"2023-09-29T23:29:39.608417+0000","last_became_active":"2023-09-29T23:29:39.604701+0000","last_became_peered":"2023-09-29T23:29:39.604701+0000","last_unstale":"2023-09-29T23:29:39.608417+0000","last_undegraded":"2023-09-29T23:29:39.608417+0000","last_fullsized":"2023-09-29T23:29:39.608417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:20:49.659581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645866+0000","last_change":"2023-09-29T23:29:39.259176+0000","last_active":"2023-09-29T23:29:39.645866+0000","last_peered":"2023-09-29T23:29:39.645866+0000","last_clean":"2023-09-29T23:29:39.645866+0000","last_became_active":"2023-09-29T23:29:39.258811+0000","last_became_peered":"2023-09-29T23:29:39.258811+0000","last_unstale":"2023-09-29T23:29:39.645866+0000","last_undegraded":"2023-09-29T23:29:39.645866+0000","last_fullsized":"2023-09-29T23:29:39.645866+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T06:38:39.409933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.608584+0000","last_change":"2023-09-29T23:29:39.606077+0000","last_active":"2023-09-29T23:29:39.608584+0000","last_peered":"2023-09-29T23:29:39.608584+0000","last_clean":"2023-09-29T23:29:39.608584+0000","last_became_active":"2023-09-29T23:29:39.605747+0000","last_became_peered":"2023-09-29T23:29:39.605747+0000","last_unstale":"2023-09-29T23:29:39.608584+0000","last_undegraded":"2023-09-29T23:29:39.608584+0000","last_fullsized":"2023-09-29T23:29:39.608584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T00:07:09.489477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"18'2","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.644653+0000","last_change":"2023-09-29T23:29:39.273703+0000","last_active":"2023-09-29T23:29:39.644653+0000","last_peered":"2023-09-29T23:29:39.644653+0000","last_clean":"2023-09-29T23:29:39.644653+0000","last_became_active":"2023-09-29T23:29:39.273385+0000","last_became_peered":"2023-09-29T23:29:39.273385+0000","last_unstale":"2023-09-29T23:29:39.644653+0000","last_undegraded":"2023-09-29T23:29:39.644653+0000","last_fullsized":"2023-09-29T23:29:39.644653+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:04:08.465658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.645244+0000","last_change":"2023-09-29T23:29:39.634607+0000","last_active":"2023-09-29T23:29:39.645244+0000","last_peered":"2023-09-29T23:29:39.645244+0000","last_clean":"2023-09-29T23:29:39.645244+0000","last_became_active":"2023-09-29T23:29:39.633666+0000","last_became_peered":"2023-09-29T23:29:39.633666+0000","last_unstale":"2023-09-29T23:29:39.645244+0000","last_undegraded":"2023-09-29T23:29:39.645244+0000","last_fullsized":"2023-09-29T23:29:39.645244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T05:19:23.819855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"18'1","reported_seq":11,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609036+0000","last_change":"2023-09-29T23:29:39.271329+0000","last_active":"2023-09-29T23:29:39.609036+0000","last_peered":"2023-09-29T23:29:39.609036+0000","last_clean":"2023-09-29T23:29:39.609036+0000","last_became_active":"2023-09-29T23:29:39.270902+0000","last_became_peered":"2023-09-29T23:29:39.270902+0000","last_unstale":"2023-09-29T23:29:39.609036+0000","last_undegraded":"2023-09-29T23:29:39.609036+0000","last_fullsized":"2023-09-29T23:29:39.609036+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T03:43:32.731713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.609490+0000","last_change":"2023-09-29T23:29:39.605447+0000","last_active":"2023-09-29T23:29:39.609490+0000","last_peered":"2023-09-29T23:29:39.609490+0000","last_clean":"2023-09-29T23:29:39.609490+0000","last_became_active":"2023-09-29T23:29:39.604800+0000","last_became_peered":"2023-09-29T23:29:39.604800+0000","last_unstale":"2023-09-29T23:29:39.609490+0000","last_undegraded":"2023-09-29T23:29:39.609490+0000","last_fullsized":"2023-09-29T23:29:39.609490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:39.227089+0000","last_clean_scrub_stamp":"2023-09-29T23:29:39.227089+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T04:58:35.776573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"18'4","reported_seq":14,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-09-29T23:29:39.631433+0000","last_change":"2023-09-29T23:29:39.251909+0000","last_active":"2023-09-29T23:29:39.631433+0000","last_peered":"2023-09-29T23:29:39.631433+0000","last_clean":"2023-09-29T23:29:39.631433+0000","last_became_active":"2023-09-29T23:29:39.251464+0000","last_became_peered":"2023-09-29T23:29:39.251464+0000","last_unstale":"2023-09-29T23:29:39.631433+0000","last_undegraded":"2023-09-29T23:29:39.631433+0000","last_fullsized":"2023-09-29T23:29:39.631433+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-29T23:29:38.223331+0000","last_clean_scrub_stamp":"2023-09-29T23:29:38.223331+0000","objects_scrubbed":0,"log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-01T08:58:28.163591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66178,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705672,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":12684,"kb_used_data":196,"kb_used_omap":0,"kb_used_meta":12480,"kb_avail":94359156,"statfs":{"total":96636764160,"available":96623775744,"internally_reserved":0,"allocated":200704,"data_stored":62103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":12779520},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70299999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":2,"up_from":9,"seq":38654705671,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13132,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358708,"statfs":{"total":96636764160,"available":96623316992,"internally_reserved":0,"allocated":266240,"data_stored":652423,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]}]},{"osd":1,"up_from":9,"seq":38654705671,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":13120,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":12864,"kb_avail":94358720,"statfs":{"total":96636764160,"available":96623329280,"internally_reserved":0,"allocated":253952,"data_stored":651387,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":13172736},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8440,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363400,"statfs":{"total":96636764160,"available":96628121600,"internally_reserved":0,"allocated":180224,"data_stored":60927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33089,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-29T23:29:50.752 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-29T23:29:50.752 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-29T23:29:50.752 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-29T23:29:50.752 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:29:50.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884f59e700 1 Processor -- start 2023-09-29T23:29:50.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884f59e700 1 -- start start 2023-09-29T23:29:50.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884f59e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f885013fce0 0x7f88500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884f59e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f88500b71e0 0x7f885014aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884f59e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88500a48e0 con 0x7f88500a5310 2023-09-29T23:29:50.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884f59e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88500b7640 con 0x7f88500b71e0 2023-09-29T23:29:50.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884f59e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88500a3de0 con 0x7f885013fce0 2023-09-29T23:29:50.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884dd9b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f885013fce0 0x7f88500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884ed9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f88500b71e0 0x7f885014aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884dd9b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f885013fce0 0x7f88500a3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60558/0 (socket says 172.21.15.143:60558) 2023-09-29T23:29:50.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884dd9b700 1 -- 172.21.15.143:0/2455641943 learned_addr learned my addr 172.21.15.143:0/2455641943 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:50.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884ed9d700 1 -- 172.21.15.143:0/2455641943 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f885013fce0 msgr2=0x7f88500a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:50.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884ed9d700 1 --2- 172.21.15.143:0/2455641943 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f885013fce0 0x7f88500a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884ed9d700 1 -- 172.21.15.143:0/2455641943 >> v1:172.21.15.156:6789/0 conn(0x7f88500a5310 legacy=0x7f885013f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.848+0000 7f884ed9d700 1 -- 172.21.15.143:0/2455641943 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88500a40a0 con 0x7f88500b71e0 2023-09-29T23:29:50.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884ed9d700 1 --2- 172.21.15.143:0/2455641943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88500b71e0 0x7f885014aa10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8848009e60 tx=0x7f8848009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6179dfa98d48c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:50.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884d59a700 1 -- 172.21.15.143:0/2455641943 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884800e070 con 0x7f88500b71e0 2023-09-29T23:29:50.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884d59a700 1 -- 172.21.15.143:0/2455641943 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884800bb60 con 0x7f88500b71e0 2023-09-29T23:29:50.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884d59a700 1 -- 172.21.15.143:0/2455641943 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8848012430 con 0x7f88500b71e0 2023-09-29T23:29:50.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 -- 172.21.15.143:0/2455641943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88500b71e0 msgr2=0x7f885014aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 --2- 172.21.15.143:0/2455641943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88500b71e0 0x7f885014aa10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8848009e60 tx=0x7f8848009780 comp rx=0 tx=0).stop 2023-09-29T23:29:50.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 -- 172.21.15.143:0/2455641943 shutdown_connections 2023-09-29T23:29:50.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 --2- 172.21.15.143:0/2455641943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88500b71e0 0x7f885014aa10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 --2- 172.21.15.143:0/2455641943 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f885013fce0 0x7f88500a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 -- 172.21.15.143:0/2455641943 >> 172.21.15.143:0/2455641943 conn(0x7f8850066c90 msgr2=0x7f8850067090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:50.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 -- 172.21.15.143:0/2455641943 shutdown_connections 2023-09-29T23:29:50.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 -- 172.21.15.143:0/2455641943 wait complete. 2023-09-29T23:29:50.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 Processor -- start 2023-09-29T23:29:50.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 -- start start 2023-09-29T23:29:50.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88500a5310 0x7f88501431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88500b71e0 0x7f88501531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f885013fce0 0x7f8850153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885009a050 con 0x7f88500b71e0 2023-09-29T23:29:50.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885006dda0 con 0x7f885013fce0 2023-09-29T23:29:50.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884f59e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885006ae70 con 0x7f88500a5310 2023-09-29T23:29:50.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884dd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88500b71e0 0x7f88501531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884e59c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88500a5310 0x7f88501431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884e59c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88500a5310 0x7f88501431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:60568/0 (socket says 172.21.15.143:60568) 2023-09-29T23:29:50.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884e59c700 1 -- 172.21.15.143:0/2104747008 learned_addr learned my addr 172.21.15.143:0/2104747008 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:50.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884ed9d700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f885013fce0 0x7f8850153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884e59c700 1 -- 172.21.15.143:0/2104747008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f885013fce0 msgr2=0x7f8850153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884e59c700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f885013fce0 0x7f8850153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884e59c700 1 -- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88500b71e0 msgr2=0x7f88501531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:50.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884e59c700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88500b71e0 0x7f88501531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:50.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.852+0000 7f884e59c700 1 -- 172.21.15.143:0/2104747008 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8848009a20 con 0x7f88500a5310 2023-09-29T23:29:50.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.856+0000 7f884e59c700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88500a5310 0x7f88501431a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f884000c7c0 tx=0x7f8840007f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:50.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.856+0000 7f883e7fc700 1 -- 172.21.15.143:0/2104747008 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840011590 con 0x7f88500a5310 2023-09-29T23:29:50.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.856+0000 7f883e7fc700 1 -- 172.21.15.143:0/2104747008 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8840011710 con 0x7f88500a5310 2023-09-29T23:29:50.859 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.856+0000 7f883e7fc700 1 -- 172.21.15.143:0/2104747008 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8840011ad0 con 0x7f88500a5310 2023-09-29T23:29:50.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.856+0000 7f884f59e700 1 -- 172.21.15.143:0/2104747008 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885006b110 con 0x7f88500a5310 2023-09-29T23:29:50.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.856+0000 7f884f59e700 1 -- 172.21.15.143:0/2104747008 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885006dac0 con 0x7f88500a5310 2023-09-29T23:29:50.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.856+0000 7f883e7fc700 1 -- 172.21.15.143:0/2104747008 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f884001c460 con 0x7f88500a5310 2023-09-29T23:29:50.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.856+0000 7f883e7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:50.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.856+0000 7f884f59e700 1 -- 172.21.15.143:0/2104747008 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885005ecb0 con 0x7f88500a5310 2023-09-29T23:29:50.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.860+0000 7f883e7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:50.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.860+0000 7f883e7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:50.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.860+0000 7f883e7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:50.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.860+0000 7f883e7fc700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f883008e860 0x7f8830090d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:50.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.860+0000 7f883e7fc700 1 -- 172.21.15.143:0/2104747008 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f88400b4490 con 0x7f88500a5310 2023-09-29T23:29:50.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.860+0000 7f884dd9b700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f883008e860 0x7f8830090d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:50.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.860+0000 7f884dd9b700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f883008e860 0x7f8830090d10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8850144600 tx=0x7f8844009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:50.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:50.864+0000 7f883e7fc700 1 -- 172.21.15.143:0/2104747008 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8840084f50 con 0x7f88500a5310 2023-09-29T23:29:51.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.060+0000 7f884f59e700 1 -- 172.21.15.143:0/2104747008 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f885006dee0 con 0x7f88500a5310 2023-09-29T23:29:51.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.060+0000 7f883e7fc700 1 -- 172.21.15.143:0/2104747008 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f885006dee0 con 0x7f88500a5310 2023-09-29T23:29:51.064 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:51.064 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:29:51.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 10 mgrc shutdown 2023-09-29T23:29:51.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 -- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f883008e860 msgr2=0x7f8830090d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:51.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f883008e860 0x7f8830090d10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8850144600 tx=0x7f8844009250 comp rx=0 tx=0).stop 2023-09-29T23:29:51.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 -- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88500a5310 msgr2=0x7f88501431a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:51.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88500a5310 0x7f88501431a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f884000c7c0 tx=0x7f8840007f10 comp rx=0 tx=0).stop 2023-09-29T23:29:51.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 -- 172.21.15.143:0/2104747008 shutdown_connections 2023-09-29T23:29:51.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f883008e860 0x7f8830090d10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:51.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f885013fce0 0x7f8850153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:51.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88500b71e0 0x7f88501531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:51.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 --2- 172.21.15.143:0/2104747008 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f88500a5310 0x7f88501431a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:51.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 -- 172.21.15.143:0/2104747008 >> 172.21.15.143:0/2104747008 conn(0x7f8850066c90 msgr2=0x7f885013d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:51.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 -- 172.21.15.143:0/2104747008 shutdown_connections 2023-09-29T23:29:51.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:51.064+0000 7f884f59e700 1 -- 172.21.15.143:0/2104747008 wait complete. 2023-09-29T23:29:54.078 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:29:54.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb663fff700 1 Processor -- start 2023-09-29T23:29:54.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb663fff700 1 -- start start 2023-09-29T23:29:54.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb663fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb664066ec0 0x7fb66406c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:54.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb663fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb66413c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:54.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb663fff700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb66406cd10 con 0x7fb66406e930 2023-09-29T23:29:54.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb663fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6640a2550 con 0x7fb664066ec0 2023-09-29T23:29:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb663fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb664067290 con 0x7fb66413c010 2023-09-29T23:29:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb6637fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb66413c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb6637fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb66413c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:42768/0 (socket says 172.21.15.143:42768) 2023-09-29T23:29:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb6637fe700 1 -- 172.21.15.143:0/1700409945 learned_addr learned my addr 172.21.15.143:0/1700409945 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb662ffd700 1 --2- 172.21.15.143:0/1700409945 >> v2:172.21.15.156:3300/0 conn(0x7fb664066ec0 0x7fb66406c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb661ffb700 1 -- 172.21.15.143:0/1700409945 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3742124803 0 0) 0x7fb66406cd10 con 0x7fb66406e930 2023-09-29T23:29:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb661ffb700 1 -- 172.21.15.143:0/1700409945 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb64c003570 con 0x7fb66406e930 2023-09-29T23:29:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb6637fe700 1 -- 172.21.15.143:0/1700409945 >> v2:172.21.15.156:3300/0 conn(0x7fb664066ec0 msgr2=0x7fb66406c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb6637fe700 1 --2- 172.21.15.143:0/1700409945 >> v2:172.21.15.156:3300/0 conn(0x7fb664066ec0 0x7fb66406c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb6637fe700 1 -- 172.21.15.143:0/1700409945 >> v1:172.21.15.143:6789/0 conn(0x7fb66406e930 legacy=0x7fb6640a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:54.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.188+0000 7fb6637fe700 1 -- 172.21.15.143:0/1700409945 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66406ad80 con 0x7fb66413c010 2023-09-29T23:29:54.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb6637fe700 1 --2- 172.21.15.143:0/1700409945 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb66413c490 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb6540099b0 tx=0x7fb654009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26c50e020b558ab server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:54.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb661ffb700 1 -- 172.21.15.143:0/1700409945 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65400b680 con 0x7fb66413c010 2023-09-29T23:29:54.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb661ffb700 1 -- 172.21.15.143:0/1700409945 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65400b7e0 con 0x7fb66413c010 2023-09-29T23:29:54.195 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb661ffb700 1 -- 172.21.15.143:0/1700409945 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65400bb60 con 0x7fb66413c010 2023-09-29T23:29:54.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb663fff700 1 -- 172.21.15.143:0/1700409945 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 msgr2=0x7fb66413c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:54.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb663fff700 1 --2- 172.21.15.143:0/1700409945 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb66413c490 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb6540099b0 tx=0x7fb654009250 comp rx=0 tx=0).stop 2023-09-29T23:29:54.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb663fff700 1 -- 172.21.15.143:0/1700409945 shutdown_connections 2023-09-29T23:29:54.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb663fff700 1 --2- 172.21.15.143:0/1700409945 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb66413c490 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:54.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb663fff700 1 --2- 172.21.15.143:0/1700409945 >> v2:172.21.15.156:3300/0 conn(0x7fb664066ec0 0x7fb66406c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:54.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb663fff700 1 -- 172.21.15.143:0/1700409945 >> 172.21.15.143:0/1700409945 conn(0x7fb66406f200 msgr2=0x7fb664069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:54.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb663fff700 1 -- 172.21.15.143:0/1700409945 shutdown_connections 2023-09-29T23:29:54.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb663fff700 1 -- 172.21.15.143:0/1700409945 wait complete. 2023-09-29T23:29:54.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb663fff700 1 Processor -- start 2023-09-29T23:29:54.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.192+0000 7fb663fff700 1 -- start start 2023-09-29T23:29:54.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb663fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664066ec0 0x7fb6641385e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:54.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb663fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb66406e930 0x7fb66413a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:54.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb663fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb6640a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:54.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb663fff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6641395d0 con 0x7fb66406e930 2023-09-29T23:29:54.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb663fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb664139dd0 con 0x7fb664066ec0 2023-09-29T23:29:54.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb663fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66414dba0 con 0x7fb66413c010 2023-09-29T23:29:54.199 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb6637fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb6640a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb6627fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb66406e930 0x7fb66413a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb6637fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb6640a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:42780/0 (socket says 172.21.15.143:42780) 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb6637fe700 1 -- 172.21.15.143:0/3815840554 learned_addr learned my addr 172.21.15.143:0/3815840554 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb662ffd700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664066ec0 0x7fb6641385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb6637fe700 1 -- 172.21.15.143:0/3815840554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664066ec0 msgr2=0x7fb6641385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb6637fe700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664066ec0 0x7fb6641385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb6637fe700 1 -- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb66406e930 msgr2=0x7fb66413a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb6637fe700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb66406e930 0x7fb66413a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb6637fe700 1 -- 172.21.15.143:0/3815840554 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6540094f0 con 0x7fb66413c010 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb6637fe700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb6640a06d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb654009290 tx=0x7fb654017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb64affd700 1 -- 172.21.15.143:0/3815840554 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb654002f90 con 0x7fb66413c010 2023-09-29T23:29:54.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb64affd700 1 -- 172.21.15.143:0/3815840554 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65401a430 con 0x7fb66413c010 2023-09-29T23:29:54.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb64affd700 1 -- 172.21.15.143:0/3815840554 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65400bb60 con 0x7fb66413c010 2023-09-29T23:29:54.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb663fff700 1 -- 172.21.15.143:0/3815840554 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb66414de60 con 0x7fb66413c010 2023-09-29T23:29:54.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.196+0000 7fb663fff700 1 -- 172.21.15.143:0/3815840554 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb66406aa00 con 0x7fb66413c010 2023-09-29T23:29:54.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.200+0000 7fb663fff700 1 -- 172.21.15.143:0/3815840554 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb664138300 con 0x7fb66413c010 2023-09-29T23:29:54.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.200+0000 7fb64affd700 1 -- 172.21.15.143:0/3815840554 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fb65401aa20 con 0x7fb66413c010 2023-09-29T23:29:54.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.200+0000 7fb64affd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.200+0000 7fb64affd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.200+0000 7fb64affd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.200+0000 7fb64affd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.200+0000 7fb64affd700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb65008e830 0x7fb650090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.200+0000 7fb662ffd700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb65008e830 0x7fb650090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.200+0000 7fb64affd700 1 -- 172.21.15.143:0/3815840554 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7fb6540b65a0 con 0x7fb66413c010 2023-09-29T23:29:54.203 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.200+0000 7fb662ffd700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb65008e830 0x7fb650090ce0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009fd0 tx=0x7fb65c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:54.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.204+0000 7fb64affd700 1 -- 172.21.15.143:0/3815840554 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb664138300 con 0x7fb66413c010 2023-09-29T23:29:54.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.396+0000 7fb663fff700 1 -- 172.21.15.143:0/3815840554 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb664138300 con 0x7fb66413c010 2023-09-29T23:29:54.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.396+0000 7fb64affd700 1 -- 172.21.15.143:0/3815840554 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fb664138300 con 0x7fb66413c010 2023-09-29T23:29:54.400 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:54.400 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:29:54.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 10 mgrc shutdown 2023-09-29T23:29:54.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 -- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb65008e830 msgr2=0x7fb650090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:54.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb65008e830 0x7fb650090ce0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009fd0 tx=0x7fb65c00b040 comp rx=0 tx=0).stop 2023-09-29T23:29:54.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 -- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 msgr2=0x7fb6640a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:54.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb6640a06d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb654009290 tx=0x7fb654017000 comp rx=0 tx=0).stop 2023-09-29T23:29:54.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 -- 172.21.15.143:0/3815840554 shutdown_connections 2023-09-29T23:29:54.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb65008e830 0x7fb650090ce0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:54.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb66413c010 0x7fb6640a06d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:54.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb66406e930 0x7fb66413a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:54.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 --2- 172.21.15.143:0/3815840554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb664066ec0 0x7fb6641385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:54.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 -- 172.21.15.143:0/3815840554 >> 172.21.15.143:0/3815840554 conn(0x7fb66406f200 msgr2=0x7fb66406fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:54.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 -- 172.21.15.143:0/3815840554 shutdown_connections 2023-09-29T23:29:54.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:54.400+0000 7fb663fff700 1 -- 172.21.15.143:0/3815840554 wait complete. 2023-09-29T23:29:55.865 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:29:55.867 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:29:57.415 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:29:57.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.524+0000 7faf17fff700 1 Processor -- start 2023-09-29T23:29:57.529 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 -- start start 2023-09-29T23:29:57.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faf1809e9e0 0x7faf1809fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:57.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faf180b34c0 0x7faf1814a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:57.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf1809f3a0 con 0x7faf18063100 2023-09-29T23:29:57.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf1809f1a0 con 0x7faf1809e9e0 2023-09-29T23:29:57.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf180634d0 con 0x7faf180b34c0 2023-09-29T23:29:57.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf167fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faf1809e9e0 0x7faf1809fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:57.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf167fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7faf1809e9e0 0x7faf1809fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38218/0 (socket says 172.21.15.143:38218) 2023-09-29T23:29:57.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf167fc700 1 -- 172.21.15.143:0/3071200507 learned_addr learned my addr 172.21.15.143:0/3071200507 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:57.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf16ffd700 1 --1- 172.21.15.143:0/3071200507 >> v1:172.21.15.143:6789/0 conn(0x7faf18063100 0x7faf1809e2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54924/0 (socket says 172.21.15.143:54924) 2023-09-29T23:29:57.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf177fe700 1 --2- 172.21.15.143:0/3071200507 >> v2:172.21.15.156:3300/0 conn(0x7faf180b34c0 0x7faf1814a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:57.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf167fc700 1 -- 172.21.15.143:0/3071200507 >> v2:172.21.15.156:3300/0 conn(0x7faf180b34c0 msgr2=0x7faf1814a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:29:57.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf167fc700 1 --2- 172.21.15.143:0/3071200507 >> v2:172.21.15.156:3300/0 conn(0x7faf180b34c0 0x7faf1814a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:57.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf167fc700 1 -- 172.21.15.143:0/3071200507 >> v1:172.21.15.143:6789/0 conn(0x7faf18063100 legacy=0x7faf1809e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:57.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf167fc700 1 -- 172.21.15.143:0/3071200507 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1809f8a0 con 0x7faf1809e9e0 2023-09-29T23:29:57.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf167fc700 1 --2- 172.21.15.143:0/3071200507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf1809e9e0 0x7faf1809fd50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faf1806a930 tx=0x7faf0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9017da61dbcfe88 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:57.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf15ffb700 1 -- 172.21.15.143:0/3071200507 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00b860 con 0x7faf1809e9e0 2023-09-29T23:29:57.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf15ffb700 1 -- 172.21.15.143:0/3071200507 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c00b9c0 con 0x7faf1809e9e0 2023-09-29T23:29:57.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf15ffb700 1 -- 172.21.15.143:0/3071200507 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00bd60 con 0x7faf1809e9e0 2023-09-29T23:29:57.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 -- 172.21.15.143:0/3071200507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf1809e9e0 msgr2=0x7faf1809fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:57.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 --2- 172.21.15.143:0/3071200507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf1809e9e0 0x7faf1809fd50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faf1806a930 tx=0x7faf0c009250 comp rx=0 tx=0).stop 2023-09-29T23:29:57.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 -- 172.21.15.143:0/3071200507 shutdown_connections 2023-09-29T23:29:57.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 --2- 172.21.15.143:0/3071200507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf1809e9e0 0x7faf1809fd50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:57.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 --2- 172.21.15.143:0/3071200507 >> v2:172.21.15.156:3300/0 conn(0x7faf180b34c0 0x7faf1814a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:57.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 -- 172.21.15.143:0/3071200507 >> 172.21.15.143:0/3071200507 conn(0x7faf1806ab30 msgr2=0x7faf1806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:57.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 -- 172.21.15.143:0/3071200507 shutdown_connections 2023-09-29T23:29:57.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.528+0000 7faf17fff700 1 -- 172.21.15.143:0/3071200507 wait complete. 2023-09-29T23:29:57.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf17fff700 1 Processor -- start 2023-09-29T23:29:57.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf17fff700 1 -- start start 2023-09-29T23:29:57.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf17fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf18063100 0x7faf18143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:57.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf17fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf1809e9e0 0x7faf18152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf17fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf180b34c0 0x7faf18142400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf17fff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf18153650 con 0x7faf1809e9e0 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf17fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf18153910 con 0x7faf180b34c0 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf17fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf18153390 con 0x7faf18063100 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf16ffd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf18063100 0x7faf18143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf167fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf1809e9e0 0x7faf18152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf167fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf1809e9e0 0x7faf18152e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38230/0 (socket says 172.21.15.143:38230) 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf167fc700 1 -- 172.21.15.143:0/2851959540 learned_addr learned my addr 172.21.15.143:0/2851959540 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf16ffd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf18063100 0x7faf18143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:42794/0 (socket says 172.21.15.143:42794) 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf177fe700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf180b34c0 0x7faf18142400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf167fc700 1 -- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf18063100 msgr2=0x7faf18143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:57.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf167fc700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf18063100 0x7faf18143da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:57.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf167fc700 1 -- 172.21.15.143:0/2851959540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf180b34c0 msgr2=0x7faf18142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:57.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf167fc700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf180b34c0 0x7faf18142400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:57.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf167fc700 1 -- 172.21.15.143:0/2851959540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0c0094f0 con 0x7faf1809e9e0 2023-09-29T23:29:57.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf177fe700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf180b34c0 0x7faf18142400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:29:57.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf167fc700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf1809e9e0 0x7faf18152e50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf1809f530 tx=0x7faf0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:57.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faefeffd700 1 -- 172.21.15.143:0/2851959540 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c024440 con 0x7faf1809e9e0 2023-09-29T23:29:57.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faefeffd700 1 -- 172.21.15.143:0/2851959540 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c0245a0 con 0x7faf1809e9e0 2023-09-29T23:29:57.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf17fff700 1 -- 172.21.15.143:0/2851959540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf18146000 con 0x7faf1809e9e0 2023-09-29T23:29:57.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faefeffd700 1 -- 172.21.15.143:0/2851959540 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c024960 con 0x7faf1809e9e0 2023-09-29T23:29:57.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.532+0000 7faf17fff700 1 -- 172.21.15.143:0/2851959540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf1809c150 con 0x7faf1809e9e0 2023-09-29T23:29:57.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.536+0000 7faf17fff700 1 -- 172.21.15.143:0/2851959540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf18136dd0 con 0x7faf1809e9e0 2023-09-29T23:29:57.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.536+0000 7faefeffd700 1 -- 172.21.15.143:0/2851959540 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7faf0c024d50 con 0x7faf1809e9e0 2023-09-29T23:29:57.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.536+0000 7faefeffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:29:57.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.540+0000 7faefeffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:29:57.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.540+0000 7faefeffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:57.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.540+0000 7faefeffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:29:57.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.540+0000 7faefeffd700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf0408e8b0 0x7faf04090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:29:57.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.540+0000 7faf16ffd700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf0408e8b0 0x7faf04090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:29:57.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.540+0000 7faefeffd700 1 -- 172.21.15.143:0/2851959540 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7faf0c0b6290 con 0x7faf1809e9e0 2023-09-29T23:29:57.542 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.540+0000 7faf16ffd700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf0408e8b0 0x7faf04090d60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf08007fd0 tx=0x7faf08009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:29:57.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.544+0000 7faefeffd700 1 -- 172.21.15.143:0/2851959540 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7faf0c086d50 con 0x7faf1809e9e0 2023-09-29T23:29:57.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.740+0000 7faf17fff700 1 -- 172.21.15.143:0/2851959540 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7faf1805ecb0 con 0x7faf1809e9e0 2023-09-29T23:29:57.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.740+0000 7faefeffd700 1 -- 172.21.15.143:0/2851959540 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7faf0c015100 con 0x7faf1809e9e0 2023-09-29T23:29:57.744 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:29:57.745 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:29:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 10 mgrc shutdown 2023-09-29T23:29:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 -- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf0408e8b0 msgr2=0x7faf04090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf0408e8b0 0x7faf04090d60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faf08007fd0 tx=0x7faf08009040 comp rx=0 tx=0).stop 2023-09-29T23:29:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 -- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf1809e9e0 msgr2=0x7faf18152e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:29:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf1809e9e0 0x7faf18152e50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7faf1809f530 tx=0x7faf0c017000 comp rx=0 tx=0).stop 2023-09-29T23:29:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 -- 172.21.15.143:0/2851959540 shutdown_connections 2023-09-29T23:29:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7faf0408e8b0 0x7faf04090d60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf180b34c0 0x7faf18142400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:57.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf1809e9e0 0x7faf18152e50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:57.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 --2- 172.21.15.143:0/2851959540 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7faf18063100 0x7faf18143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:29:57.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 -- 172.21.15.143:0/2851959540 >> 172.21.15.143:0/2851959540 conn(0x7faf1806ab30 msgr2=0x7faf180b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:29:57.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 -- 172.21.15.143:0/2851959540 shutdown_connections 2023-09-29T23:29:57.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:29:57.744+0000 7faf17fff700 1 -- 172.21.15.143:0/2851959540 wait complete. 2023-09-29T23:29:59.999 INFO:tasks.ceph.mon.a.smithi143.stderr:2023-09-29T23:29:59.995+0000 7fdae4109700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-29T23:30:00.759 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:00.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.867+0000 7f6de8e84700 1 Processor -- start 2023-09-29T23:30:00.875 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.867+0000 7f6de8e84700 1 -- start start 2023-09-29T23:30:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.867+0000 7f6de8e84700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6de40a1170 0x7f6de413d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.867+0000 7f6de8e84700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de409ff40 con 0x7f6de413d680 2023-09-29T23:30:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.867+0000 7f6de8e84700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de4067290 con 0x7f6de4066ec0 2023-09-29T23:30:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.867+0000 7f6de8e84700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de40a0510 con 0x7f6de40a1170 2023-09-29T23:30:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.867+0000 7f6de3fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f6de413d680 0x7f6de4148590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:54938/0 (socket says 172.21.15.143:54938) 2023-09-29T23:30:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.867+0000 7f6de3fff700 1 -- 172.21.15.143:0/2837389175 learned_addr learned my addr 172.21.15.143:0/2837389175 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:30:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.867+0000 7f6de2ffd700 1 --2- 172.21.15.143:0/2837389175 >> v2:172.21.15.156:3300/0 conn(0x7f6de40a1170 0x7f6de413d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.867+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231378832 0 0) 0x7f6de409ff40 con 0x7f6de413d680 2023-09-29T23:30:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.867+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6dd0003570 con 0x7f6de413d680 2023-09-29T23:30:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1945796592 0 0) 0x7f6dd0003570 con 0x7f6de413d680 2023-09-29T23:30:00.876 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 >> v2:172.21.15.156:3300/0 conn(0x7f6de40a1170 msgr2=0x7f6de413d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 --2- 172.21.15.143:0/2837389175 >> v2:172.21.15.156:3300/0 conn(0x7f6de40a1170 0x7f6de413d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 >> v1:172.21.15.156:6789/0 conn(0x7f6de4066ec0 legacy=0x7f6de409f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de406e400 con 0x7f6de413d680 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251153714 0 0) 0x7f6ddc001470 con 0x7f6de413d680 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 >> v1:172.21.15.143:6789/0 conn(0x7f6de413d680 legacy=0x7f6de4148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 --2- 172.21.15.143:0/2837389175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dd0003a60 0x7f6dd0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 --2- 172.21.15.143:0/2837389175 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6dd0004810 0x7f6dd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 --2- 172.21.15.143:0/2837389175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dd00051c0 0x7f6dd000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de40a0510 con 0x7f6dd0003a60 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd000c050 con 0x7f6dd00051c0 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd000c210 con 0x7f6dd0004810 2023-09-29T23:30:00.877 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de37fe700 1 --2- 172.21.15.143:0/2837389175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dd0003a60 0x7f6dd0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de3fff700 1 --2- 172.21.15.143:0/2837389175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dd00051c0 0x7f6dd000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de2ffd700 1 --2- 172.21.15.143:0/2837389175 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6dd0004810 0x7f6dd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de37fe700 1 -- 172.21.15.143:0/2837389175 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6dd0004810 msgr2=0x7f6dd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de37fe700 1 --2- 172.21.15.143:0/2837389175 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6dd0004810 0x7f6dd0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de37fe700 1 -- 172.21.15.143:0/2837389175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dd00051c0 msgr2=0x7f6dd000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de37fe700 1 --2- 172.21.15.143:0/2837389175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dd00051c0 0x7f6dd000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de37fe700 1 -- 172.21.15.143:0/2837389175 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd0003570 con 0x7f6dd0003a60 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de3fff700 1 -- 172.21.15.143:0/2837389175 reap_dead start 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de37fe700 1 --2- 172.21.15.143:0/2837389175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dd0003a60 0x7f6dd0003e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0019a0 tx=0x7f6dcc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a87d1c4ab9cdc00 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc00c5d0 con 0x7f6dd0003a60 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc00c750 con 0x7f6dd0003a60 2023-09-29T23:30:00.878 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de27fc700 1 -- 172.21.15.143:0/2837389175 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc00cb10 con 0x7f6dd0003a60 2023-09-29T23:30:00.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de8e84700 1 -- 172.21.15.143:0/2837389175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dd0003a60 msgr2=0x7f6dd0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:00.879 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de8e84700 1 --2- 172.21.15.143:0/2837389175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dd0003a60 0x7f6dd0003e30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc0019a0 tx=0x7f6dcc007d10 comp rx=0 tx=0).stop 2023-09-29T23:30:00.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de8e84700 1 -- 172.21.15.143:0/2837389175 shutdown_connections 2023-09-29T23:30:00.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de8e84700 1 --2- 172.21.15.143:0/2837389175 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6dd0003a60 0x7f6dd0003e30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:00.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de8e84700 1 -- 172.21.15.143:0/2837389175 >> 172.21.15.143:0/2837389175 conn(0x7f6de406f200 msgr2=0x7f6de4069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:00.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de8e84700 1 -- 172.21.15.143:0/2837389175 shutdown_connections 2023-09-29T23:30:00.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.871+0000 7f6de8e84700 1 -- 172.21.15.143:0/2837389175 wait complete. 2023-09-29T23:30:00.881 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de8e84700 1 Processor -- start 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de8e84700 1 -- start start 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de8e84700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6dd0003a60 0x7f6de413a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de8e84700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de4140120 0x7f6de4139cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de8e84700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6de4098970 0x7f6de4098d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de8e84700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4139310 con 0x7f6de4140120 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de8e84700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de413a200 con 0x7f6de4098970 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de8e84700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4140510 con 0x7f6dd0003a60 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de37fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6dd0003a60 0x7f6de413a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de37fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6dd0003a60 0x7f6de413a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:42802/0 (socket says 172.21.15.143:42802) 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de37fe700 1 -- 172.21.15.143:0/4080802158 learned_addr learned my addr 172.21.15.143:0/4080802158 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de2ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de4140120 0x7f6de4139cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:00.882 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de3fff700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6de4098970 0x7f6de4098d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de37fe700 1 -- 172.21.15.143:0/4080802158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6de4098970 msgr2=0x7f6de4098d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de37fe700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6de4098970 0x7f6de4098d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de37fe700 1 -- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de4140120 msgr2=0x7f6de4139cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de37fe700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de4140120 0x7f6de4139cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de37fe700 1 -- 172.21.15.143:0/4080802158 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de41407c0 con 0x7f6dd0003a60 2023-09-29T23:30:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de37fe700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6dd0003a60 0x7f6de413a640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc012e40 tx=0x7f6dcc015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6ddb7fe700 1 -- 172.21.15.143:0/4080802158 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc00ce80 con 0x7f6dd0003a60 2023-09-29T23:30:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6ddb7fe700 1 -- 172.21.15.143:0/4080802158 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc00c8c0 con 0x7f6dd0003a60 2023-09-29T23:30:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de8e84700 1 -- 172.21.15.143:0/4080802158 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de4140a80 con 0x7f6dd0003a60 2023-09-29T23:30:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6ddb7fe700 1 -- 172.21.15.143:0/4080802158 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc026400 con 0x7f6dd0003a60 2023-09-29T23:30:00.883 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.875+0000 7f6de8e84700 1 -- 172.21.15.143:0/4080802158 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de4138670 con 0x7f6dd0003a60 2023-09-29T23:30:00.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.879+0000 7f6de8e84700 1 -- 172.21.15.143:0/4080802158 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de405ecb0 con 0x7f6dd0003a60 2023-09-29T23:30:00.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.879+0000 7f6ddb7fe700 1 -- 172.21.15.143:0/4080802158 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f6dcc019030 con 0x7f6dd0003a60 2023-09-29T23:30:00.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.879+0000 7f6ddb7fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:00.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.879+0000 7f6ddb7fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:00.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.879+0000 7f6ddb7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:00.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.879+0000 7f6ddb7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.879+0000 7f6ddb7fe700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f6dc408e8b0 0x7f6dc4090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.879+0000 7f6ddb7fe700 1 -- 172.21.15.143:0/4080802158 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f6dcc0b5aa0 con 0x7f6dd0003a60 2023-09-29T23:30:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.879+0000 7f6de2ffd700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f6dc408e8b0 0x7f6dc4090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:00.885 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.883+0000 7f6de2ffd700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f6dc408e8b0 0x7f6dc4090d60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6de4141f70 tx=0x7f6dd400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:00.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:00.891+0000 7f6ddb7fe700 1 -- 172.21.15.143:0/4080802158 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f6dcc086620 con 0x7f6dd0003a60 2023-09-29T23:30:01.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.083+0000 7f6de8e84700 1 -- 172.21.15.143:0/4080802158 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6de406e6c0 con 0x7f6dd0003a60 2023-09-29T23:30:01.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6ddb7fe700 1 -- 172.21.15.143:0/4080802158 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f6dcc02b100 con 0x7f6dd0003a60 2023-09-29T23:30:01.090 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:01.090 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 10 mgrc shutdown 2023-09-29T23:30:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 -- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f6dc408e8b0 msgr2=0x7f6dc4090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f6dc408e8b0 0x7f6dc4090d60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6de4141f70 tx=0x7f6dd400b040 comp rx=0 tx=0).stop 2023-09-29T23:30:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 -- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6dd0003a60 msgr2=0x7f6de413a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6dd0003a60 0x7f6de413a640 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc012e40 tx=0x7f6dcc015000 comp rx=0 tx=0).stop 2023-09-29T23:30:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 -- 172.21.15.143:0/4080802158 shutdown_connections 2023-09-29T23:30:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f6dc408e8b0 0x7f6dc4090d60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6de4098970 0x7f6de4098d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:01.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6de4140120 0x7f6de4139cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:01.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 --2- 172.21.15.143:0/4080802158 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f6dd0003a60 0x7f6de413a640 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:01.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 -- 172.21.15.143:0/4080802158 >> 172.21.15.143:0/4080802158 conn(0x7f6de406f200 msgr2=0x7f6de413ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:01.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 -- 172.21.15.143:0/4080802158 shutdown_connections 2023-09-29T23:30:01.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:01.087+0000 7f6de8e84700 1 -- 172.21.15.143:0/4080802158 wait complete. 2023-09-29T23:30:04.104 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:04.214 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295bb13700 1 Processor -- start 2023-09-29T23:30:04.215 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295bb13700 1 -- start start 2023-09-29T23:30:04.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295bb13700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f29540b3140 0x7f29540a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:04.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295bb13700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f295413d620 0x7f295413da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:04.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295bb13700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f29540a7930 0x7f29540a7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:04.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295bb13700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29540a82b0 con 0x7f295413d620 2023-09-29T23:30:04.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295bb13700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29540b3510 con 0x7f29540b3140 2023-09-29T23:30:04.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295bb13700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2954141710 con 0x7f29540a7930 2023-09-29T23:30:04.216 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f29590ae700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f295413d620 0x7f295413da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:04.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295a0b0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f29540a7930 0x7f29540a7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:04.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f29590ae700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f295413d620 0x7f295413da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46538/0 (socket says 172.21.15.143:46538) 2023-09-29T23:30:04.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f29590ae700 1 -- 172.21.15.143:0/2680445403 learned_addr learned my addr 172.21.15.143:0/2680445403 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:04.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f29598af700 1 --2- 172.21.15.143:0/2680445403 >> v2:172.21.15.156:3300/0 conn(0x7f29540b3140 0x7f29540a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:04.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295a0b0700 1 -- 172.21.15.143:0/2680445403 >> v2:172.21.15.156:3300/0 conn(0x7f29540b3140 msgr2=0x7f29540a61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:04.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295a0b0700 1 --2- 172.21.15.143:0/2680445403 >> v2:172.21.15.156:3300/0 conn(0x7f29540b3140 0x7f29540a61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295a0b0700 1 -- 172.21.15.143:0/2680445403 >> v2:172.21.15.143:3300/0 conn(0x7f295413d620 msgr2=0x7f295413da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:04.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295a0b0700 1 --2- 172.21.15.143:0/2680445403 >> v2:172.21.15.143:3300/0 conn(0x7f295413d620 0x7f295413da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295a0b0700 1 -- 172.21.15.143:0/2680445403 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29541419d0 con 0x7f29540a7930 2023-09-29T23:30:04.217 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f29590ae700 1 --2- 172.21.15.143:0/2680445403 >> v2:172.21.15.143:3300/0 conn(0x7f295413d620 0x7f295413da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:30:04.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f295a0b0700 1 --2- 172.21.15.143:0/2680445403 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f29540a7930 0x7f29540a7d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2954141bc0 tx=0x7f294c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad4e215c93aab932 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:04.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.211+0000 7f29588ad700 1 -- 172.21.15.143:0/2680445403 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c011630 con 0x7f29540a7930 2023-09-29T23:30:04.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29588ad700 1 -- 172.21.15.143:0/2680445403 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c011790 con 0x7f29540a7930 2023-09-29T23:30:04.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29588ad700 1 -- 172.21.15.143:0/2680445403 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c011b50 con 0x7f29540a7930 2023-09-29T23:30:04.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 -- 172.21.15.143:0/2680445403 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f29540a7930 msgr2=0x7f29540a7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:04.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 --2- 172.21.15.143:0/2680445403 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f29540a7930 0x7f29540a7d70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2954141bc0 tx=0x7f294c0091a0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 -- 172.21.15.143:0/2680445403 shutdown_connections 2023-09-29T23:30:04.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 --2- 172.21.15.143:0/2680445403 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f29540a7930 0x7f29540a7d70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 --2- 172.21.15.143:0/2680445403 >> v2:172.21.15.143:3300/0 conn(0x7f295413d620 0x7f295413da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 --2- 172.21.15.143:0/2680445403 >> v2:172.21.15.156:3300/0 conn(0x7f29540b3140 0x7f29540a61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.219 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 -- 172.21.15.143:0/2680445403 >> 172.21.15.143:0/2680445403 conn(0x7f2954067dc0 msgr2=0x7f29540b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:04.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 -- 172.21.15.143:0/2680445403 shutdown_connections 2023-09-29T23:30:04.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 -- 172.21.15.143:0/2680445403 wait complete. 2023-09-29T23:30:04.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 Processor -- start 2023-09-29T23:30:04.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 -- start start 2023-09-29T23:30:04.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29540a7930 0x7f29540af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:04.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29540b3140 0x7f295411af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:04.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f295413d620 0x7f2954153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:04.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295411b4b0 con 0x7f29540b3140 2023-09-29T23:30:04.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29540add50 con 0x7f29540a7930 2023-09-29T23:30:04.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295bb13700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29540adff0 con 0x7f295413d620 2023-09-29T23:30:04.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29590ae700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29540b3140 0x7f295411af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:04.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f295a0b0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f295413d620 0x7f2954153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:04.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29590ae700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29540b3140 0x7f295411af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46550/0 (socket says 172.21.15.143:46550) 2023-09-29T23:30:04.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29590ae700 1 -- 172.21.15.143:0/1726485515 learned_addr learned my addr 172.21.15.143:0/1726485515 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:04.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29598af700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29540a7930 0x7f29540af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:04.222 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29590ae700 1 -- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f295413d620 msgr2=0x7f2954153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:04.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29590ae700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f295413d620 0x7f2954153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29590ae700 1 -- 172.21.15.143:0/1726485515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29540a7930 msgr2=0x7f29540af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:04.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29590ae700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29540a7930 0x7f29540af470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29590ae700 1 -- 172.21.15.143:0/1726485515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2950009ae0 con 0x7f29540b3140 2023-09-29T23:30:04.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f29590ae700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29540b3140 0x7f295411af70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f295000c170 tx=0x7f295000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:04.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.215+0000 7f2949ffb700 1 -- 172.21.15.143:0/1726485515 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295000caa0 con 0x7f29540b3140 2023-09-29T23:30:04.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.219+0000 7f295bb13700 1 -- 172.21.15.143:0/1726485515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c009420 con 0x7f29540b3140 2023-09-29T23:30:04.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.219+0000 7f2949ffb700 1 -- 172.21.15.143:0/1726485515 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295000f040 con 0x7f29540b3140 2023-09-29T23:30:04.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.219+0000 7f2949ffb700 1 -- 172.21.15.143:0/1726485515 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295000ce30 con 0x7f29540b3140 2023-09-29T23:30:04.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.219+0000 7f295bb13700 1 -- 172.21.15.143:0/1726485515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2954153540 con 0x7f29540b3140 2023-09-29T23:30:04.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.219+0000 7f295bb13700 1 -- 172.21.15.143:0/1726485515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2954153700 con 0x7f29540b3140 2023-09-29T23:30:04.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.219+0000 7f2949ffb700 1 -- 172.21.15.143:0/1726485515 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f2950014690 con 0x7f29540b3140 2023-09-29T23:30:04.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.219+0000 7f2949ffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:04.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.219+0000 7f2949ffb700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:04.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.219+0000 7f2949ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:04.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.219+0000 7f2949ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:04.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.219+0000 7f2949ffb700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f294008e830 0x7f2940090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:04.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.223+0000 7f29598af700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f294008e830 0x7f2940090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:04.225 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.223+0000 7f2949ffb700 1 -- 172.21.15.143:0/1726485515 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f29500b4f40 con 0x7f29540b3140 2023-09-29T23:30:04.226 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.223+0000 7f29598af700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f294008e830 0x7f2940090ce0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2944009db0 tx=0x7f29440091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:04.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.227+0000 7f2949ffb700 1 -- 172.21.15.143:0/1726485515 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2950085970 con 0x7f29540b3140 2023-09-29T23:30:04.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.423+0000 7f295bb13700 1 -- 172.21.15.143:0/1726485515 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f295405ecb0 con 0x7f29540b3140 2023-09-29T23:30:04.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.423+0000 7f2949ffb700 1 -- 172.21.15.143:0/1726485515 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f2950088f90 con 0x7f29540b3140 2023-09-29T23:30:04.428 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:04.428 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:04.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 10 mgrc shutdown 2023-09-29T23:30:04.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 -- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f294008e830 msgr2=0x7f2940090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:04.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f294008e830 0x7f2940090ce0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2944009db0 tx=0x7f29440091a0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 -- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29540b3140 msgr2=0x7f295411af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:04.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29540b3140 0x7f295411af70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f295000c170 tx=0x7f295000b040 comp rx=0 tx=0).stop 2023-09-29T23:30:04.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 -- 172.21.15.143:0/1726485515 shutdown_connections 2023-09-29T23:30:04.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f294008e830 0x7f2940090ce0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f295413d620 0x7f2954153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29540b3140 0x7f295411af70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 --2- 172.21.15.143:0/1726485515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29540a7930 0x7f29540af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:04.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 -- 172.21.15.143:0/1726485515 >> 172.21.15.143:0/1726485515 conn(0x7f2954067dc0 msgr2=0x7f295413e210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:04.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 -- 172.21.15.143:0/1726485515 shutdown_connections 2023-09-29T23:30:04.431 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:04.427+0000 7f295bb13700 1 -- 172.21.15.143:0/1726485515 wait complete. 2023-09-29T23:30:07.443 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:07.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.551+0000 7f7531aeb700 1 Processor -- start 2023-09-29T23:30:07.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.551+0000 7f7531aeb700 1 -- start start 2023-09-29T23:30:07.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f752c06efd0 0x7f752c09fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:07.557 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 0x7f752c0b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:07.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f752c148270 con 0x7f752c09e510 2023-09-29T23:30:07.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f752c148530 con 0x7f752c06efd0 2023-09-29T23:30:07.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c06f3a0 con 0x7f752c0b20d0 2023-09-29T23:30:07.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f75312ea700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 0x7f752c0b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:07.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f752bfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f752c09e510 0x7f752c147990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:38002/0 (socket says 172.21.15.143:38002) 2023-09-29T23:30:07.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f752bfff700 1 -- 172.21.15.143:0/2199667683 learned_addr learned my addr 172.21.15.143:0/2199667683 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:30:07.558 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7530ae9700 1 --2- 172.21.15.143:0/2199667683 >> v2:172.21.15.156:3300/0 conn(0x7f752c06efd0 0x7f752c09fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:07.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f752b7fe700 1 -- 172.21.15.143:0/2199667683 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694160136 0 0) 0x7f752c148270 con 0x7f752c09e510 2023-09-29T23:30:07.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f752b7fe700 1 -- 172.21.15.143:0/2199667683 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f751c003570 con 0x7f752c09e510 2023-09-29T23:30:07.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f75312ea700 1 -- 172.21.15.143:0/2199667683 >> v2:172.21.15.156:3300/0 conn(0x7f752c06efd0 msgr2=0x7f752c09fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:07.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f75312ea700 1 --2- 172.21.15.143:0/2199667683 >> v2:172.21.15.156:3300/0 conn(0x7f752c06efd0 0x7f752c09fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:07.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f75312ea700 1 -- 172.21.15.143:0/2199667683 >> v1:172.21.15.143:6789/0 conn(0x7f752c09e510 legacy=0x7f752c147990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:07.560 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f752b7fe700 1 -- 172.21.15.143:0/2199667683 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2516156045 0 0) 0x7f751c003570 con 0x7f752c09e510 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f75312ea700 1 -- 172.21.15.143:0/2199667683 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752c0a0180 con 0x7f752c0b20d0 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f75312ea700 1 --2- 172.21.15.143:0/2199667683 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 0x7f752c0b2550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7524009e60 tx=0x7f7524009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c38d49add397677 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f752b7fe700 1 -- 172.21.15.143:0/2199667683 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400e070 con 0x7f752c0b20d0 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f752b7fe700 1 -- 172.21.15.143:0/2199667683 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752400b7a0 con 0x7f752c0b20d0 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f752b7fe700 1 -- 172.21.15.143:0/2199667683 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518001470 con 0x7f752c0b20d0 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 -- 172.21.15.143:0/2199667683 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 msgr2=0x7f752c0b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 --2- 172.21.15.143:0/2199667683 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 0x7f752c0b2550 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7524009e60 tx=0x7f7524009780 comp rx=0 tx=0).stop 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 -- 172.21.15.143:0/2199667683 shutdown_connections 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 --2- 172.21.15.143:0/2199667683 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 0x7f752c0b2550 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 --2- 172.21.15.143:0/2199667683 >> v2:172.21.15.156:3300/0 conn(0x7f752c06efd0 0x7f752c09fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 -- 172.21.15.143:0/2199667683 >> 172.21.15.143:0/2199667683 conn(0x7f752c0b2ca0 msgr2=0x7f752c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:07.561 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 -- 172.21.15.143:0/2199667683 shutdown_connections 2023-09-29T23:30:07.562 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.555+0000 7f7531aeb700 1 -- 172.21.15.143:0/2199667683 wait complete. 2023-09-29T23:30:07.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7531aeb700 1 Processor -- start 2023-09-29T23:30:07.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7531aeb700 1 -- start start 2023-09-29T23:30:07.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7531aeb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f752c06efd0 0x7f752c0a4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:07.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7531aeb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c09e510 0x7f752c06c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:07.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7531aeb700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 0x7f752c1444f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:07.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7531aeb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c0afe80 con 0x7f752c09e510 2023-09-29T23:30:07.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7531aeb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752c06bfe0 con 0x7f752c06efd0 2023-09-29T23:30:07.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7531aeb700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752c0af1b0 con 0x7f752c0b20d0 2023-09-29T23:30:07.564 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f75312ea700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 0x7f752c1444f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f75312ea700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 0x7f752c1444f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:34130/0 (socket says 172.21.15.143:34130) 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f75312ea700 1 -- 172.21.15.143:0/1721391283 learned_addr learned my addr 172.21.15.143:0/1721391283 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f752bfff700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c09e510 0x7f752c06c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7530ae9700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f752c06efd0 0x7f752c0a4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f75312ea700 1 -- 172.21.15.143:0/1721391283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f752c06efd0 msgr2=0x7f752c0a4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f75312ea700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f752c06efd0 0x7f752c0a4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f75312ea700 1 -- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c09e510 msgr2=0x7f752c06c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f75312ea700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c09e510 0x7f752c06c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f75312ea700 1 -- 172.21.15.143:0/1721391283 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524009a20 con 0x7f752c0b20d0 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f75312ea700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 0x7f752c1444f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f752c06f4e0 tx=0x7f7524018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7528ff9700 1 -- 172.21.15.143:0/1721391283 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7524012d20 con 0x7f752c0b20d0 2023-09-29T23:30:07.565 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7528ff9700 1 -- 172.21.15.143:0/1721391283 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752401e430 con 0x7f752c0b20d0 2023-09-29T23:30:07.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7531aeb700 1 -- 172.21.15.143:0/1721391283 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c0af4b0 con 0x7f752c0b20d0 2023-09-29T23:30:07.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7528ff9700 1 -- 172.21.15.143:0/1721391283 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752401e7f0 con 0x7f752c0b20d0 2023-09-29T23:30:07.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.559+0000 7f7531aeb700 1 -- 172.21.15.143:0/1721391283 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f752c06bd00 con 0x7f752c0b20d0 2023-09-29T23:30:07.566 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.563+0000 7f7531aeb700 1 -- 172.21.15.143:0/1721391283 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f752c06c4e0 con 0x7f752c0b20d0 2023-09-29T23:30:07.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.563+0000 7f7528ff9700 1 -- 172.21.15.143:0/1721391283 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f752401eb60 con 0x7f752c0b20d0 2023-09-29T23:30:07.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.563+0000 7f7528ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:07.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.567+0000 7f7528ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:07.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.567+0000 7f7528ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:07.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.567+0000 7f7528ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:07.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.567+0000 7f7528ff9700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751008e8b0 0x7f7510090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:07.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.567+0000 7f7528ff9700 1 -- 172.21.15.143:0/1721391283 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f75240b6e60 con 0x7f752c0b20d0 2023-09-29T23:30:07.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.567+0000 7f7530ae9700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751008e8b0 0x7f7510090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:07.570 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.567+0000 7f7530ae9700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751008e8b0 0x7f7510090d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7520009d40 tx=0x7f7520008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:07.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.571+0000 7f7528ff9700 1 -- 172.21.15.143:0/1721391283 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7524087920 con 0x7f752c0b20d0 2023-09-29T23:30:07.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.763+0000 7f7531aeb700 1 -- 172.21.15.143:0/1721391283 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f752c0af4b0 con 0x7f752c0b20d0 2023-09-29T23:30:07.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.763+0000 7f7528ff9700 1 -- 172.21.15.143:0/1721391283 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f752c0af4b0 con 0x7f752c0b20d0 2023-09-29T23:30:07.768 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:07.768 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:07.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 10 mgrc shutdown 2023-09-29T23:30:07.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 -- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751008e8b0 msgr2=0x7f7510090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:07.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751008e8b0 0x7f7510090d60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7520009d40 tx=0x7f7520008040 comp rx=0 tx=0).stop 2023-09-29T23:30:07.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 -- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 msgr2=0x7f752c1444f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:07.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 0x7f752c1444f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f752c06f4e0 tx=0x7f7524018000 comp rx=0 tx=0).stop 2023-09-29T23:30:07.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 -- 172.21.15.143:0/1721391283 shutdown_connections 2023-09-29T23:30:07.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751008e8b0 0x7f7510090d60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:07.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f752c0b20d0 0x7f752c1444f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:07.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f752c09e510 0x7f752c06c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:07.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 --2- 172.21.15.143:0/1721391283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f752c06efd0 0x7f752c0a4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:07.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 -- 172.21.15.143:0/1721391283 >> 172.21.15.143:0/1721391283 conn(0x7f752c0b2ca0 msgr2=0x7f752c0683b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:07.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 -- 172.21.15.143:0/1721391283 shutdown_connections 2023-09-29T23:30:07.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:07.767+0000 7f7531aeb700 1 -- 172.21.15.143:0/1721391283 wait complete. 2023-09-29T23:30:10.786 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:10.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.895+0000 7febe531c700 1 Processor -- start 2023-09-29T23:30:10.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 -- start start 2023-09-29T23:30:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 0x7febe00b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7febe00a1680 0x7febe00a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe00a07e0 con 0x7febe00a0f50 2023-09-29T23:30:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe01410d0 con 0x7febe00a1680 2023-09-29T23:30:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe00a42c0 con 0x7febe0066ec0 2023-09-29T23:30:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe4b1b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7febe00a1680 0x7febe00a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe4b1b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7febe00a1680 0x7febe00a5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46576/0 (socket says 172.21.15.143:46576) 2023-09-29T23:30:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe4b1b700 1 -- 172.21.15.143:0/1613036095 learned_addr learned my addr 172.21.15.143:0/1613036095 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:10.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febdffff700 1 --2- 172.21.15.143:0/1613036095 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 0x7febe00b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:10.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe4b1b700 1 -- 172.21.15.143:0/1613036095 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 msgr2=0x7febe00b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:10.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe4b1b700 1 --2- 172.21.15.143:0/1613036095 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 0x7febe00b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:10.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe4b1b700 1 -- 172.21.15.143:0/1613036095 >> v1:172.21.15.156:6789/0 conn(0x7febe00a0f50 legacy=0x7febe00a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:10.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe4b1b700 1 -- 172.21.15.143:0/1613036095 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe00a4580 con 0x7febe00a1680 2023-09-29T23:30:10.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe4b1b700 1 --2- 172.21.15.143:0/1613036095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febe00a1680 0x7febe00a5ef0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7febd8009f20 tx=0x7febd8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecc798c4951df9c server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:10.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febdeffd700 1 -- 172.21.15.143:0/1613036095 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd800e070 con 0x7febe00a1680 2023-09-29T23:30:10.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febdeffd700 1 -- 172.21.15.143:0/1613036095 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd800b9d0 con 0x7febe00a1680 2023-09-29T23:30:10.904 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febdeffd700 1 -- 172.21.15.143:0/1613036095 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd80092e0 con 0x7febe00a1680 2023-09-29T23:30:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 -- 172.21.15.143:0/1613036095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febe00a1680 msgr2=0x7febe00a5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 --2- 172.21.15.143:0/1613036095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febe00a1680 0x7febe00a5ef0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7febd8009f20 tx=0x7febd8009840 comp rx=0 tx=0).stop 2023-09-29T23:30:10.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 -- 172.21.15.143:0/1613036095 shutdown_connections 2023-09-29T23:30:10.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 --2- 172.21.15.143:0/1613036095 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febe00a1680 0x7febe00a5ef0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:10.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 --2- 172.21.15.143:0/1613036095 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 0x7febe00b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:10.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 -- 172.21.15.143:0/1613036095 >> 172.21.15.143:0/1613036095 conn(0x7febe0067da0 msgr2=0x7febe00681a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:10.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 -- 172.21.15.143:0/1613036095 shutdown_connections 2023-09-29T23:30:10.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.899+0000 7febe531c700 1 -- 172.21.15.143:0/1613036095 wait complete. 2023-09-29T23:30:10.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febe531c700 1 Processor -- start 2023-09-29T23:30:10.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febe531c700 1 -- start start 2023-09-29T23:30:10.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febe531c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 0x7febe0153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:10.909 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febe531c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febe00a0f50 0x7febe013d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:10.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febe531c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe00a1680 0x7febe0143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:10.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febe531c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe00b1650 con 0x7febe00a0f50 2023-09-29T23:30:10.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febe531c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe013d0b0 con 0x7febe00a1680 2023-09-29T23:30:10.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febe531c700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe0153cd0 con 0x7febe0066ec0 2023-09-29T23:30:10.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febdf7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febe00a0f50 0x7febe013d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:10.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febdffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 0x7febe0153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:10.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febdf7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febe00a0f50 0x7febe013d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46592/0 (socket says 172.21.15.143:46592) 2023-09-29T23:30:10.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febdf7fe700 1 -- 172.21.15.143:0/1495118318 learned_addr learned my addr 172.21.15.143:0/1495118318 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:10.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febdffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 0x7febe0153620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:34162/0 (socket says 172.21.15.143:34162) 2023-09-29T23:30:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febe4b1b700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe00a1680 0x7febe0143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febdffff700 1 -- 172.21.15.143:0/1495118318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe00a1680 msgr2=0x7febe0143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febdffff700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe00a1680 0x7febe0143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febdffff700 1 -- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febe00a0f50 msgr2=0x7febe013d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febdffff700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febe00a0f50 0x7febe013d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.903+0000 7febdffff700 1 -- 172.21.15.143:0/1495118318 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd8009ae0 con 0x7febe0066ec0 2023-09-29T23:30:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.907+0000 7febdffff700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 0x7febe0153620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7febd000e930 tx=0x7febd00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.907+0000 7febc7fff700 1 -- 172.21.15.143:0/1495118318 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0013680 con 0x7febe0066ec0 2023-09-29T23:30:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.907+0000 7febc7fff700 1 -- 172.21.15.143:0/1495118318 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd00137e0 con 0x7febe0066ec0 2023-09-29T23:30:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.907+0000 7febc7fff700 1 -- 172.21.15.143:0/1495118318 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0013b80 con 0x7febe0066ec0 2023-09-29T23:30:10.911 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.907+0000 7febe531c700 1 -- 172.21.15.143:0/1495118318 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe0153f90 con 0x7febe0066ec0 2023-09-29T23:30:10.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.907+0000 7febe531c700 1 -- 172.21.15.143:0/1495118318 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe00abcf0 con 0x7febe0066ec0 2023-09-29T23:30:10.912 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.907+0000 7febe531c700 1 -- 172.21.15.143:0/1495118318 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe005ecb0 con 0x7febe0066ec0 2023-09-29T23:30:10.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.911+0000 7febc7fff700 1 -- 172.21.15.143:0/1495118318 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7febd00253d0 con 0x7febe0066ec0 2023-09-29T23:30:10.913 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.911+0000 7febc7fff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:10.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.911+0000 7febc7fff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:10.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.911+0000 7febc7fff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:10.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.911+0000 7febc7fff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:10.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.911+0000 7febc7fff700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7febcc08e8e0 0x7febcc090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:10.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.911+0000 7febdf7fe700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7febcc08e8e0 0x7febcc090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:10.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.911+0000 7febc7fff700 1 -- 172.21.15.143:0/1495118318 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7febd00b4fe0 con 0x7febe0066ec0 2023-09-29T23:30:10.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.911+0000 7febdf7fe700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7febcc08e8e0 0x7febcc090d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febe0141660 tx=0x7febd4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:10.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:10.915+0000 7febc7fff700 1 -- 172.21.15.143:0/1495118318 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7febd0085a40 con 0x7febe0066ec0 2023-09-29T23:30:11.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.107+0000 7febe531c700 1 -- 172.21.15.143:0/1495118318 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7febe0067290 con 0x7febe0066ec0 2023-09-29T23:30:11.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febc7fff700 1 -- 172.21.15.143:0/1495118318 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7febd0017100 con 0x7febe0066ec0 2023-09-29T23:30:11.113 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:11.113 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:11.115 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 10 mgrc shutdown 2023-09-29T23:30:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 -- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7febcc08e8e0 msgr2=0x7febcc090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7febcc08e8e0 0x7febcc090d90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7febe0141660 tx=0x7febd4009040 comp rx=0 tx=0).stop 2023-09-29T23:30:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 -- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 msgr2=0x7febe0153620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 0x7febe0153620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7febd000e930 tx=0x7febd00091a0 comp rx=0 tx=0).stop 2023-09-29T23:30:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 -- 172.21.15.143:0/1495118318 shutdown_connections 2023-09-29T23:30:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7febcc08e8e0 0x7febcc090d90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe00a1680 0x7febe0143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7febe00a0f50 0x7febe013d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 --2- 172.21.15.143:0/1495118318 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7febe0066ec0 0x7febe0153620 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 -- 172.21.15.143:0/1495118318 >> 172.21.15.143:0/1495118318 conn(0x7febe0067da0 msgr2=0x7febe00b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:11.116 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 -- 172.21.15.143:0/1495118318 shutdown_connections 2023-09-29T23:30:11.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:11.111+0000 7febe531c700 1 -- 172.21.15.143:0/1495118318 wait complete. 2023-09-29T23:30:14.127 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:14.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b759e700 1 Processor -- start 2023-09-29T23:30:14.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b759e700 1 -- start start 2023-09-29T23:30:14.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b759e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b8066ec0 0x7fd2b809fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:14.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b759e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2b806b680 0x7fd2b80a0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:14.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b759e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b8067290 con 0x7fd2b809f450 2023-09-29T23:30:14.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b759e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b80a0b40 con 0x7fd2b806b680 2023-09-29T23:30:14.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b759e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b8067ad0 con 0x7fd2b8066ec0 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b659c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b8066ec0 0x7fd2b809fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b5d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2b806b680 0x7fd2b80a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b659c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b8066ec0 0x7fd2b809fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:45190/0 (socket says 172.21.15.143:45190) 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b659c700 1 -- 172.21.15.143:0/1213262718 learned_addr learned my addr 172.21.15.143:0/1213262718 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b5d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2b806b680 0x7fd2b80a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37566/0 (socket says 172.21.15.143:37566) 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b5d9b700 1 -- 172.21.15.143:0/1213262718 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b8066ec0 msgr2=0x7fd2b809fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b5d9b700 1 --2- 172.21.15.143:0/1213262718 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b8066ec0 0x7fd2b809fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b5d9b700 1 -- 172.21.15.143:0/1213262718 >> v1:172.21.15.156:6789/0 conn(0x7fd2b809f450 legacy=0x7fd2b8135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b5d9b700 1 -- 172.21.15.143:0/1213262718 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b80a0dc0 con 0x7fd2b806b680 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b659c700 1 --2- 172.21.15.143:0/1213262718 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b8066ec0 0x7fd2b809fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b5d9b700 1 --2- 172.21.15.143:0/1213262718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2b806b680 0x7fd2b80a0190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009e60 tx=0x7fd2a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8449f9b6982ce57e server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:14.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.235+0000 7fd2b559a700 1 -- 172.21.15.143:0/1213262718 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a800e070 con 0x7fd2b806b680 2023-09-29T23:30:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b559a700 1 -- 172.21.15.143:0/1213262718 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a800bb60 con 0x7fd2b806b680 2023-09-29T23:30:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b559a700 1 -- 172.21.15.143:0/1213262718 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8012430 con 0x7fd2b806b680 2023-09-29T23:30:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b759e700 1 -- 172.21.15.143:0/1213262718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2b806b680 msgr2=0x7fd2b80a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b759e700 1 --2- 172.21.15.143:0/1213262718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2b806b680 0x7fd2b80a0190 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009e60 tx=0x7fd2a8009780 comp rx=0 tx=0).stop 2023-09-29T23:30:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b759e700 1 -- 172.21.15.143:0/1213262718 shutdown_connections 2023-09-29T23:30:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b759e700 1 --2- 172.21.15.143:0/1213262718 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2b806b680 0x7fd2b80a0190 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b759e700 1 --2- 172.21.15.143:0/1213262718 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b8066ec0 0x7fd2b809fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b759e700 1 -- 172.21.15.143:0/1213262718 >> 172.21.15.143:0/1213262718 conn(0x7fd2b806f200 msgr2=0x7fd2b8069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b759e700 1 -- 172.21.15.143:0/1213262718 shutdown_connections 2023-09-29T23:30:14.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b759e700 1 -- 172.21.15.143:0/1213262718 wait complete. 2023-09-29T23:30:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b759e700 1 Processor -- start 2023-09-29T23:30:14.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.239+0000 7fd2b759e700 1 -- start start 2023-09-29T23:30:14.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b759e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2b8066ec0 0x7fd2b8137a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:14.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b759e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b806b680 0x7fd2b809d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:14.246 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b759e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2b809f450 0x7fd2b809d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b759e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b809deb0 con 0x7fd2b809f450 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b759e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b809e0c0 con 0x7fd2b8066ec0 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b759e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b809e380 con 0x7fd2b806b680 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b5d9b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b806b680 0x7fd2b809d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b6d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2b809f450 0x7fd2b809d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b5d9b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b806b680 0x7fd2b809d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:45192/0 (socket says 172.21.15.143:45192) 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b5d9b700 1 -- 172.21.15.143:0/3994756662 learned_addr learned my addr 172.21.15.143:0/3994756662 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2b8066ec0 0x7fd2b8137a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b6d9d700 1 -- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b806b680 msgr2=0x7fd2b809d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b6d9d700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b806b680 0x7fd2b809d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b6d9d700 1 -- 172.21.15.143:0/3994756662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2b8066ec0 msgr2=0x7fd2b8137a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b6d9d700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2b8066ec0 0x7fd2b8137a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:14.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b6d9d700 1 -- 172.21.15.143:0/3994756662 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a00094f0 con 0x7fd2b809f450 2023-09-29T23:30:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b6d9d700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2b809f450 0x7fd2b809d970 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000c4b0 tx=0x7fd2b0007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2ae7fc700 1 -- 172.21.15.143:0/3994756662 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000ce60 con 0x7fd2b809f450 2023-09-29T23:30:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2ae7fc700 1 -- 172.21.15.143:0/3994756662 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b0011590 con 0x7fd2b809f450 2023-09-29T23:30:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2ae7fc700 1 -- 172.21.15.143:0/3994756662 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0011950 con 0x7fd2b809f450 2023-09-29T23:30:14.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b759e700 1 -- 172.21.15.143:0/3994756662 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a8009a20 con 0x7fd2b809f450 2023-09-29T23:30:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.243+0000 7fd2b759e700 1 -- 172.21.15.143:0/3994756662 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b80a1c20 con 0x7fd2b809f450 2023-09-29T23:30:14.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.247+0000 7fd2b759e700 1 -- 172.21.15.143:0/3994756662 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b809d130 con 0x7fd2b809f450 2023-09-29T23:30:14.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.247+0000 7fd2ae7fc700 1 -- 172.21.15.143:0/3994756662 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fd2b0011cc0 con 0x7fd2b809f450 2023-09-29T23:30:14.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.247+0000 7fd2ae7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:14.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.247+0000 7fd2ae7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:14.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.247+0000 7fd2ae7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:14.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.247+0000 7fd2ae7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:14.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.247+0000 7fd2ae7fc700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd29808e830 0x7fd298090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:14.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.247+0000 7fd2b659c700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd29808e830 0x7fd298090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:14.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.247+0000 7fd2ae7fc700 1 -- 172.21.15.143:0/3994756662 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7fd2b00b3b70 con 0x7fd2b809f450 2023-09-29T23:30:14.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.251+0000 7fd2b659c700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd29808e830 0x7fd298090ce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0006d20 tx=0x7fd2a0009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:14.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.255+0000 7fd2ae7fc700 1 -- 172.21.15.143:0/3994756662 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd2b0084630 con 0x7fd2b809f450 2023-09-29T23:30:14.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.447+0000 7fd2b759e700 1 -- 172.21.15.143:0/3994756662 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd2b805ecb0 con 0x7fd2b809f450 2023-09-29T23:30:14.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.447+0000 7fd2ae7fc700 1 -- 172.21.15.143:0/3994756662 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fd2b0017020 con 0x7fd2b809f450 2023-09-29T23:30:14.451 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:14.452 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:14.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 10 mgrc shutdown 2023-09-29T23:30:14.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 -- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd29808e830 msgr2=0x7fd298090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:14.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd29808e830 0x7fd298090ce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0006d20 tx=0x7fd2a0009250 comp rx=0 tx=0).stop 2023-09-29T23:30:14.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 -- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2b809f450 msgr2=0x7fd2b809d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:14.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2b809f450 0x7fd2b809d970 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd2b000c4b0 tx=0x7fd2b0007e60 comp rx=0 tx=0).stop 2023-09-29T23:30:14.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 -- 172.21.15.143:0/3994756662 shutdown_connections 2023-09-29T23:30:14.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd29808e830 0x7fd298090ce0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:14.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2b809f450 0x7fd2b809d970 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:14.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2b806b680 0x7fd2b809d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:14.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 --2- 172.21.15.143:0/3994756662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2b8066ec0 0x7fd2b8137a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:14.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 -- 172.21.15.143:0/3994756662 >> 172.21.15.143:0/3994756662 conn(0x7fd2b806f200 msgr2=0x7fd2b8135a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:14.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 -- 172.21.15.143:0/3994756662 shutdown_connections 2023-09-29T23:30:14.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:14.451+0000 7fd2b759e700 1 -- 172.21.15.143:0/3994756662 wait complete. 2023-09-29T23:30:17.466 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:17.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c5a694700 1 Processor -- start 2023-09-29T23:30:17.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c5a694700 1 -- start start 2023-09-29T23:30:17.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c5a694700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7c5406de00 0x7f7c54140880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:17.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c5a694700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c540a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:17.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c5a694700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c5406e1d0 con 0x7f7c540a6720 2023-09-29T23:30:17.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c5a694700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c541402a0 con 0x7f7c5406de00 2023-09-29T23:30:17.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c5a694700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5409e2d0 con 0x7f7c5406a2a0 2023-09-29T23:30:17.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c59e93700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c540a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:17.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c59e93700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c540a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:45198/0 (socket says 172.21.15.143:45198) 2023-09-29T23:30:17.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c59e93700 1 -- 172.21.15.143:0/3306411248 learned_addr learned my addr 172.21.15.143:0/3306411248 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:17.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c59692700 1 --2- 172.21.15.143:0/3306411248 >> v2:172.21.15.156:3300/0 conn(0x7f7c5406de00 0x7f7c54140880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:17.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c59e93700 1 -- 172.21.15.143:0/3306411248 >> v2:172.21.15.156:3300/0 conn(0x7f7c5406de00 msgr2=0x7f7c54140880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:17.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c59e93700 1 --2- 172.21.15.143:0/3306411248 >> v2:172.21.15.156:3300/0 conn(0x7f7c5406de00 0x7f7c54140880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:17.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c59e93700 1 -- 172.21.15.143:0/3306411248 >> v1:172.21.15.156:6789/0 conn(0x7f7c540a6720 legacy=0x7f7c54069b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:17.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c59e93700 1 -- 172.21.15.143:0/3306411248 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5413ff90 con 0x7f7c5406a2a0 2023-09-29T23:30:17.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c59e93700 1 --2- 172.21.15.143:0/3306411248 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c540a6af0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0099b0 tx=0x7f7c4c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=308280f1cdb5d103 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:17.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c4bfff700 1 -- 172.21.15.143:0/3306411248 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c009ea0 con 0x7f7c5406a2a0 2023-09-29T23:30:17.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c4bfff700 1 -- 172.21.15.143:0/3306411248 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4c00b860 con 0x7f7c5406a2a0 2023-09-29T23:30:17.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.579+0000 7f7c4bfff700 1 -- 172.21.15.143:0/3306411248 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4c00bc00 con 0x7f7c5406a2a0 2023-09-29T23:30:17.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 -- 172.21.15.143:0/3306411248 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 msgr2=0x7f7c540a6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:17.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 --2- 172.21.15.143:0/3306411248 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c540a6af0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c0099b0 tx=0x7f7c4c009250 comp rx=0 tx=0).stop 2023-09-29T23:30:17.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 -- 172.21.15.143:0/3306411248 shutdown_connections 2023-09-29T23:30:17.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 --2- 172.21.15.143:0/3306411248 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c540a6af0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:17.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 --2- 172.21.15.143:0/3306411248 >> v2:172.21.15.156:3300/0 conn(0x7f7c5406de00 0x7f7c54140880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:17.586 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 -- 172.21.15.143:0/3306411248 >> 172.21.15.143:0/3306411248 conn(0x7f7c540b4640 msgr2=0x7f7c540b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:17.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 -- 172.21.15.143:0/3306411248 shutdown_connections 2023-09-29T23:30:17.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 -- 172.21.15.143:0/3306411248 wait complete. 2023-09-29T23:30:17.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 Processor -- start 2023-09-29T23:30:17.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 -- start start 2023-09-29T23:30:17.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c5409ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:17.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c5406de00 0x7f7c54153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:17.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540a6720 0x7f7c54153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:17.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5409c0b0 con 0x7f7c540a6720 2023-09-29T23:30:17.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c540a3ff0 con 0x7f7c5406de00 2023-09-29T23:30:17.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c5a694700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5409cf90 con 0x7f7c5406a2a0 2023-09-29T23:30:17.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c59692700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c5409ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:17.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c59692700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c5409ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:45202/0 (socket says 172.21.15.143:45202) 2023-09-29T23:30:17.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c59e93700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540a6720 0x7f7c54153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:17.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c59692700 1 -- 172.21.15.143:0/139768238 learned_addr learned my addr 172.21.15.143:0/139768238 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:17.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c58e91700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c5406de00 0x7f7c54153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:17.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c59692700 1 -- 172.21.15.143:0/139768238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c5406de00 msgr2=0x7f7c54153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:17.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c59692700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c5406de00 0x7f7c54153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:17.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c59692700 1 -- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540a6720 msgr2=0x7f7c54153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:17.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c59692700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540a6720 0x7f7c54153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:17.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.583+0000 7f7c59692700 1 -- 172.21.15.143:0/139768238 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c0094f0 con 0x7f7c5406a2a0 2023-09-29T23:30:17.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.587+0000 7f7c58e91700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c5406de00 0x7f7c54153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:30:17.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.587+0000 7f7c59e93700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540a6720 0x7f7c54153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:30:17.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.587+0000 7f7c59692700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c5409ca50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400ec80 tx=0x7f7c440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:17.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.587+0000 7f7c497fa700 1 -- 172.21.15.143:0/139768238 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4400c850 con 0x7f7c5406a2a0 2023-09-29T23:30:17.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.587+0000 7f7c497fa700 1 -- 172.21.15.143:0/139768238 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c4400c9b0 con 0x7f7c5406a2a0 2023-09-29T23:30:17.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.587+0000 7f7c5a694700 1 -- 172.21.15.143:0/139768238 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5409a530 con 0x7f7c5406a2a0 2023-09-29T23:30:17.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.587+0000 7f7c497fa700 1 -- 172.21.15.143:0/139768238 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c4400cd50 con 0x7f7c5406a2a0 2023-09-29T23:30:17.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.587+0000 7f7c5a694700 1 -- 172.21.15.143:0/139768238 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c54099fa0 con 0x7f7c5406a2a0 2023-09-29T23:30:17.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.587+0000 7f7c5a694700 1 -- 172.21.15.143:0/139768238 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5405ecb0 con 0x7f7c5406a2a0 2023-09-29T23:30:17.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.591+0000 7f7c497fa700 1 -- 172.21.15.143:0/139768238 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f7c44013680 con 0x7f7c5406a2a0 2023-09-29T23:30:17.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.591+0000 7f7c497fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:17.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.591+0000 7f7c497fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:17.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.591+0000 7f7c497fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:17.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.591+0000 7f7c497fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:17.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.591+0000 7f7c497fa700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c3808e830 0x7f7c38090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:17.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.591+0000 7f7c58e91700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c3808e830 0x7f7c38090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:17.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.591+0000 7f7c497fa700 1 -- 172.21.15.143:0/139768238 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f7c440b51c0 con 0x7f7c5406a2a0 2023-09-29T23:30:17.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.591+0000 7f7c58e91700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c3808e830 0x7f7c38090ce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c540a0ea0 tx=0x7f7c50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:17.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.595+0000 7f7c497fa700 1 -- 172.21.15.143:0/139768238 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7c44085c80 con 0x7f7c5406a2a0 2023-09-29T23:30:17.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.787+0000 7f7c5a694700 1 -- 172.21.15.143:0/139768238 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7c5409e5a0 con 0x7f7c5406a2a0 2023-09-29T23:30:17.792 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.787+0000 7f7c497fa700 1 -- 172.21.15.143:0/139768238 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f7c44017100 con 0x7f7c5406a2a0 2023-09-29T23:30:17.793 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:17.793 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:17.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 10 mgrc shutdown 2023-09-29T23:30:17.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 -- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c3808e830 msgr2=0x7f7c38090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:17.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c3808e830 0x7f7c38090ce0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c540a0ea0 tx=0x7f7c50009040 comp rx=0 tx=0).stop 2023-09-29T23:30:17.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 -- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 msgr2=0x7f7c5409ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:17.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c5409ca50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c4400ec80 tx=0x7f7c440091a0 comp rx=0 tx=0).stop 2023-09-29T23:30:17.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 -- 172.21.15.143:0/139768238 shutdown_connections 2023-09-29T23:30:17.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c3808e830 0x7f7c38090ce0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:17.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c540a6720 0x7f7c54153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:17.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c5406de00 0x7f7c54153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:17.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 --2- 172.21.15.143:0/139768238 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c5406a2a0 0x7f7c5409ca50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:17.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 -- 172.21.15.143:0/139768238 >> 172.21.15.143:0/139768238 conn(0x7f7c540b4640 msgr2=0x7f7c5413e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:17.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 -- 172.21.15.143:0/139768238 shutdown_connections 2023-09-29T23:30:17.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:17.791+0000 7f7c5a694700 1 -- 172.21.15.143:0/139768238 wait complete. 2023-09-29T23:30:20.807 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:20.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.915+0000 7f318ab19700 1 Processor -- start 2023-09-29T23:30:20.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.915+0000 7f318ab19700 1 -- start start 2023-09-29T23:30:20.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.915+0000 7f318ab19700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318413c720 0x7f318406c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.915+0000 7f318ab19700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31840a2550 con 0x7f318406e930 2023-09-29T23:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.915+0000 7f318ab19700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f318406cd10 con 0x7f3184066ec0 2023-09-29T23:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.915+0000 7f318ab19700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3184067290 con 0x7f318413c720 2023-09-29T23:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.915+0000 7f3189316700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f318406e930 0x7f318413c010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46964/0 (socket says 172.21.15.143:46964) 2023-09-29T23:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.915+0000 7f3189316700 1 -- 172.21.15.143:0/2711071232 learned_addr learned my addr 172.21.15.143:0/2711071232 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.915+0000 7f318a318700 1 --2- 172.21.15.143:0/2711071232 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318413c720 0x7f318406c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.915+0000 7f3188b15700 1 -- 172.21.15.143:0/2711071232 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365391115 0 0) 0x7f31840a2550 con 0x7f318406e930 2023-09-29T23:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.915+0000 7f3188b15700 1 -- 172.21.15.143:0/2711071232 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3170003570 con 0x7f318406e930 2023-09-29T23:30:20.921 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318a318700 1 -- 172.21.15.143:0/2711071232 >> v1:172.21.15.156:6789/0 conn(0x7f3184066ec0 legacy=0x7f31840a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318a318700 1 -- 172.21.15.143:0/2711071232 >> v1:172.21.15.143:6789/0 conn(0x7f318406e930 legacy=0x7f318413c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:20.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318a318700 1 -- 172.21.15.143:0/2711071232 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318406ad80 con 0x7f318413c720 2023-09-29T23:30:20.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318a318700 1 --2- 172.21.15.143:0/2711071232 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318413c720 0x7f318406c7d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f317c009e90 tx=0x7f317c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11de5ca7f3dcee7 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:20.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f3188b15700 1 -- 172.21.15.143:0/2711071232 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c00e070 con 0x7f318413c720 2023-09-29T23:30:20.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f3188b15700 1 -- 172.21.15.143:0/2711071232 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c00bc30 con 0x7f318413c720 2023-09-29T23:30:20.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f3188b15700 1 -- 172.21.15.143:0/2711071232 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c012430 con 0x7f318413c720 2023-09-29T23:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 -- 172.21.15.143:0/2711071232 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318413c720 msgr2=0x7f318406c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 --2- 172.21.15.143:0/2711071232 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318413c720 0x7f318406c7d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f317c009e90 tx=0x7f317c009780 comp rx=0 tx=0).stop 2023-09-29T23:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 -- 172.21.15.143:0/2711071232 shutdown_connections 2023-09-29T23:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 --2- 172.21.15.143:0/2711071232 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318413c720 0x7f318406c7d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 -- 172.21.15.143:0/2711071232 >> 172.21.15.143:0/2711071232 conn(0x7f318406f200 msgr2=0x7f3184069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 -- 172.21.15.143:0/2711071232 shutdown_connections 2023-09-29T23:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 -- 172.21.15.143:0/2711071232 wait complete. 2023-09-29T23:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 Processor -- start 2023-09-29T23:30:20.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 -- start start 2023-09-29T23:30:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3184066ec0 0x7f318413fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318406e930 0x7f318409c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318413c720 0x7f318414ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:20.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3184097a50 con 0x7f318413c720 2023-09-29T23:30:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31841403e0 con 0x7f3184066ec0 2023-09-29T23:30:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318ab19700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3184153400 con 0x7f318406e930 2023-09-29T23:30:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f3189316700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318406e930 0x7f318409c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318a318700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318413c720 0x7f318414ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f3189316700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318406e930 0x7f318409c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:45218/0 (socket says 172.21.15.143:45218) 2023-09-29T23:30:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f3189316700 1 -- 172.21.15.143:0/3134914713 learned_addr learned my addr 172.21.15.143:0/3134914713 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f3189b17700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3184066ec0 0x7f318413fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318a318700 1 -- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318406e930 msgr2=0x7f318409c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318a318700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318406e930 0x7f318409c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:20.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318a318700 1 -- 172.21.15.143:0/3134914713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3184066ec0 msgr2=0x7f318413fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318a318700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3184066ec0 0x7f318413fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.919+0000 7f318a318700 1 -- 172.21.15.143:0/3134914713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c009a20 con 0x7f318413c720 2023-09-29T23:30:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.923+0000 7f318a318700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318413c720 0x7f318414ec30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f317c016000 tx=0x7f317c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.923+0000 7f3179ffb700 1 -- 172.21.15.143:0/3134914713 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c012c50 con 0x7f318413c720 2023-09-29T23:30:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.923+0000 7f3179ffb700 1 -- 172.21.15.143:0/3134914713 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c021070 con 0x7f318413c720 2023-09-29T23:30:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.923+0000 7f318ab19700 1 -- 172.21.15.143:0/3134914713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31841536c0 con 0x7f318413c720 2023-09-29T23:30:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.923+0000 7f3179ffb700 1 -- 172.21.15.143:0/3134914713 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c026430 con 0x7f318413c720 2023-09-29T23:30:20.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.923+0000 7f318ab19700 1 -- 172.21.15.143:0/3134914713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3184096b40 con 0x7f318413c720 2023-09-29T23:30:20.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.923+0000 7f318ab19700 1 -- 172.21.15.143:0/3134914713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318409c060 con 0x7f318413c720 2023-09-29T23:30:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.927+0000 7f3179ffb700 1 -- 172.21.15.143:0/3134914713 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f317c023030 con 0x7f318413c720 2023-09-29T23:30:20.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.927+0000 7f3179ffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:20.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.927+0000 7f3179ffb700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:20.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.927+0000 7f3179ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:20.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.927+0000 7f3179ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:20.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.927+0000 7f3179ffb700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f316808e8b0 0x7f3168090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:20.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.927+0000 7f3189b17700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f316808e8b0 0x7f3168090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:20.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.927+0000 7f3179ffb700 1 -- 172.21.15.143:0/3134914713 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 4330+0+0 (secure 0 0 0) 0x7f317c0b6a40 con 0x7f318413c720 2023-09-29T23:30:20.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.927+0000 7f3189b17700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f316808e8b0 0x7f3168090d60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3180003aa0 tx=0x7f3180000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:20.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:20.931+0000 7f3179ffb700 1 -- 172.21.15.143:0/3134914713 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f318409c060 con 0x7f318413c720 2023-09-29T23:30:21.130 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.127+0000 7f318ab19700 1 -- 172.21.15.143:0/3134914713 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f318409c060 con 0x7f318413c720 2023-09-29T23:30:21.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.127+0000 7f3179ffb700 1 -- 172.21.15.143:0/3134914713 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f318409c060 con 0x7f318413c720 2023-09-29T23:30:21.132 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:21.132 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:21.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 10 mgrc shutdown 2023-09-29T23:30:21.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 -- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f316808e8b0 msgr2=0x7f3168090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:21.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f316808e8b0 0x7f3168090d60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3180003aa0 tx=0x7f3180000bc0 comp rx=0 tx=0).stop 2023-09-29T23:30:21.134 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 -- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318413c720 msgr2=0x7f318414ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:21.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318413c720 0x7f318414ec30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f317c016000 tx=0x7f317c019000 comp rx=0 tx=0).stop 2023-09-29T23:30:21.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 -- 172.21.15.143:0/3134914713 shutdown_connections 2023-09-29T23:30:21.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f316808e8b0 0x7f3168090d60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:21.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f318413c720 0x7f318414ec30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:21.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f318406e930 0x7f318409c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:21.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 --2- 172.21.15.143:0/3134914713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3184066ec0 0x7f318413fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:21.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 -- 172.21.15.143:0/3134914713 >> 172.21.15.143:0/3134914713 conn(0x7f318406f200 msgr2=0x7f318413bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:21.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 -- 172.21.15.143:0/3134914713 shutdown_connections 2023-09-29T23:30:21.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:21.131+0000 7f318ab19700 1 -- 172.21.15.143:0/3134914713 wait complete. 2023-09-29T23:30:24.146 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:24.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.243+0000 7f8a4d2e1700 1 Processor -- start 2023-09-29T23:30:24.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.243+0000 7f8a4d2e1700 1 -- start start 2023-09-29T23:30:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4d2e1700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8a4806ee30 0x7f8a480a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4d2e1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a4809bee0 0x7f8a4809c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4d2e1700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a48140200 con 0x7f8a4806f200 2023-09-29T23:30:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4d2e1700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a481404c0 con 0x7f8a4809bee0 2023-09-29T23:30:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4d2e1700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a4809f950 con 0x7f8a4806ee30 2023-09-29T23:30:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4cae0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8a4809bee0 0x7f8a4809c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:24.250 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a3ffff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8a4806f200 0x7f8a48140780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56096/0 (socket says 172.21.15.143:56096) 2023-09-29T23:30:24.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a3ffff700 1 -- 172.21.15.143:0/3876693204 learned_addr learned my addr 172.21.15.143:0/3876693204 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:30:24.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a47fff700 1 --2- 172.21.15.143:0/3876693204 >> v2:172.21.15.156:3300/0 conn(0x7f8a4806ee30 0x7f8a480a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:24.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4cae0700 1 -- 172.21.15.143:0/3876693204 >> v2:172.21.15.156:3300/0 conn(0x7f8a4806ee30 msgr2=0x7f8a480a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:24.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4cae0700 1 --2- 172.21.15.143:0/3876693204 >> v2:172.21.15.156:3300/0 conn(0x7f8a4806ee30 0x7f8a480a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:24.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4cae0700 1 -- 172.21.15.143:0/3876693204 >> v1:172.21.15.143:6789/0 conn(0x7f8a4806f200 legacy=0x7f8a48140780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:24.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a477fe700 1 -- 172.21.15.143:0/3876693204 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1609495139 0 0) 0x7f8a48140200 con 0x7f8a4806f200 2023-09-29T23:30:24.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4cae0700 1 -- 172.21.15.143:0/3876693204 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a48140e90 con 0x7f8a4809bee0 2023-09-29T23:30:24.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4cae0700 1 --2- 172.21.15.143:0/3876693204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4809bee0 0x7f8a4809c380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a4809ca20 tx=0x7f8a40009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a5a41112d9866c server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:24.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a477fe700 1 -- 172.21.15.143:0/3876693204 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4000e070 con 0x7f8a4809bee0 2023-09-29T23:30:24.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a477fe700 1 -- 172.21.15.143:0/3876693204 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4000bb60 con 0x7f8a4809bee0 2023-09-29T23:30:24.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a477fe700 1 -- 172.21.15.143:0/3876693204 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40012430 con 0x7f8a4809bee0 2023-09-29T23:30:24.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/3876693204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4809bee0 msgr2=0x7f8a4809c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:24.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4d2e1700 1 --2- 172.21.15.143:0/3876693204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4809bee0 0x7f8a4809c380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a4809ca20 tx=0x7f8a40009780 comp rx=0 tx=0).stop 2023-09-29T23:30:24.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/3876693204 shutdown_connections 2023-09-29T23:30:24.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4d2e1700 1 --2- 172.21.15.143:0/3876693204 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4809bee0 0x7f8a4809c380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:24.253 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4d2e1700 1 --2- 172.21.15.143:0/3876693204 >> v2:172.21.15.156:3300/0 conn(0x7f8a4806ee30 0x7f8a480a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:24.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.247+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/3876693204 >> 172.21.15.143:0/3876693204 conn(0x7f8a4806a470 msgr2=0x7f8a4806a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:24.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/3876693204 shutdown_connections 2023-09-29T23:30:24.254 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/3876693204 wait complete. 2023-09-29T23:30:24.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4d2e1700 1 Processor -- start 2023-09-29T23:30:24.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4d2e1700 1 -- start start 2023-09-29T23:30:24.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4d2e1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4806ee30 0x7f8a480a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:24.257 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4d2e1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4806f200 0x7f8a48143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4d2e1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8a4809bee0 0x7f8a48143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4d2e1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4806deb0 con 0x7f8a4806ee30 2023-09-29T23:30:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4d2e1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48144280 con 0x7f8a4806f200 2023-09-29T23:30:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4d2e1700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4806e010 con 0x7f8a4809bee0 2023-09-29T23:30:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a47fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4806ee30 0x7f8a480a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4cae0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8a4809bee0 0x7f8a48143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4cae0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8a4809bee0 0x7f8a48143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43946/0 (socket says 172.21.15.143:43946) 2023-09-29T23:30:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a4cae0700 1 -- 172.21.15.143:0/1892450991 learned_addr learned my addr 172.21.15.143:0/1892450991 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.251+0000 7f8a3ffff700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4806f200 0x7f8a48143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a4cae0700 1 -- 172.21.15.143:0/1892450991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4806f200 msgr2=0x7f8a48143720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:24.258 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a4cae0700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4806f200 0x7f8a48143720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a4cae0700 1 -- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4806ee30 msgr2=0x7f8a480a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a4cae0700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4806ee30 0x7f8a480a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a4cae0700 1 -- 172.21.15.143:0/1892450991 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a40009a20 con 0x7f8a4809bee0 2023-09-29T23:30:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a47fff700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4806ee30 0x7f8a480a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:30:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a4cae0700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8a4809bee0 0x7f8a48143c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8a4809cc80 tx=0x7f8a40019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a44ff9700 1 -- 172.21.15.143:0/1892450991 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40027470 con 0x7f8a4809bee0 2023-09-29T23:30:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a44ff9700 1 -- 172.21.15.143:0/1892450991 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a400275f0 con 0x7f8a4809bee0 2023-09-29T23:30:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/1892450991 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4806d1e0 con 0x7f8a4809bee0 2023-09-29T23:30:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a44ff9700 1 -- 172.21.15.143:0/1892450991 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a400279b0 con 0x7f8a4809bee0 2023-09-29T23:30:24.259 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/1892450991 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a480a42c0 con 0x7f8a4809bee0 2023-09-29T23:30:24.260 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.255+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/1892450991 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4806e1a0 con 0x7f8a4809bee0 2023-09-29T23:30:24.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.259+0000 7f8a44ff9700 1 -- 172.21.15.143:0/1892450991 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f8a40024070 con 0x7f8a4809bee0 2023-09-29T23:30:24.261 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.259+0000 7f8a44ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:24.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.259+0000 7f8a44ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:24.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.259+0000 7f8a44ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:24.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.259+0000 7f8a44ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:24.262 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.259+0000 7f8a44ff9700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a2808e930 0x7f8a28090de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:24.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.259+0000 7f8a47fff700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a2808e930 0x7f8a28090de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:24.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.259+0000 7f8a44ff9700 1 -- 172.21.15.143:0/1892450991 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f8a400b75e0 con 0x7f8a4809bee0 2023-09-29T23:30:24.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.259+0000 7f8a47fff700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a2808e930 0x7f8a28090de0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a380113e0 tx=0x7f8a380091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:24.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.263+0000 7f8a44ff9700 1 -- 172.21.15.143:0/1892450991 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8a40087f90 con 0x7f8a4809bee0 2023-09-29T23:30:24.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.455+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/1892450991 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8a4805ecb0 con 0x7f8a4809bee0 2023-09-29T23:30:24.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.455+0000 7f8a44ff9700 1 -- 172.21.15.143:0/1892450991 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f8a4008b5b0 con 0x7f8a4809bee0 2023-09-29T23:30:24.459 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:24.459 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:24.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 10 mgrc shutdown 2023-09-29T23:30:24.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a2808e930 msgr2=0x7f8a28090de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:24.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a2808e930 0x7f8a28090de0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a380113e0 tx=0x7f8a380091a0 comp rx=0 tx=0).stop 2023-09-29T23:30:24.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8a4809bee0 msgr2=0x7f8a48143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:24.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8a4809bee0 0x7f8a48143c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8a4809cc80 tx=0x7f8a40019000 comp rx=0 tx=0).stop 2023-09-29T23:30:24.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/1892450991 shutdown_connections 2023-09-29T23:30:24.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a2808e930 0x7f8a28090de0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:24.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8a4809bee0 0x7f8a48143c60 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:24.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a4806f200 0x7f8a48143720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:24.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 --2- 172.21.15.143:0/1892450991 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8a4806ee30 0x7f8a480a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:24.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/1892450991 >> 172.21.15.143:0/1892450991 conn(0x7f8a4806a470 msgr2=0x7f8a480a1570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:24.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/1892450991 shutdown_connections 2023-09-29T23:30:24.463 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:24.459+0000 7f8a4d2e1700 1 -- 172.21.15.143:0/1892450991 wait complete. 2023-09-29T23:30:25.912 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:30:25.915 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:30:27.474 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:27.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.583+0000 7f7489c71700 1 Processor -- start 2023-09-29T23:30:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.583+0000 7f7489c71700 1 -- start start 2023-09-29T23:30:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.583+0000 7f7489c71700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 0x7f748406c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:27.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.583+0000 7f7489c71700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74840a2550 con 0x7f7484066ec0 2023-09-29T23:30:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.583+0000 7f7489c71700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748406cd10 con 0x7f748413c010 2023-09-29T23:30:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.583+0000 7f7489c71700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7484067290 con 0x7f748406e930 2023-09-29T23:30:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.583+0000 7f7483fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 0x7f748406c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.583+0000 7f7488c6f700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7484066ec0 0x7f74840a1e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56112/0 (socket says 172.21.15.143:56112) 2023-09-29T23:30:27.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.583+0000 7f7488c6f700 1 -- 172.21.15.143:0/1983290436 learned_addr learned my addr 172.21.15.143:0/1983290436 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:30:27.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f74837fe700 1 -- 172.21.15.143:0/1983290436 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359163358 0 0) 0x7f74840a2550 con 0x7f7484066ec0 2023-09-29T23:30:27.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f74837fe700 1 -- 172.21.15.143:0/1983290436 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f746c003570 con 0x7f7484066ec0 2023-09-29T23:30:27.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7483fff700 1 -- 172.21.15.143:0/1983290436 >> v1:172.21.15.156:6789/0 conn(0x7f748413c010 legacy=0x7f748413c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:27.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7483fff700 1 -- 172.21.15.143:0/1983290436 >> v1:172.21.15.143:6789/0 conn(0x7f7484066ec0 legacy=0x7f74840a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:27.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7483fff700 1 -- 172.21.15.143:0/1983290436 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748406ad80 con 0x7f748406e930 2023-09-29T23:30:27.590 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7483fff700 1 --2- 172.21.15.143:0/1983290436 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 0x7f748406c7d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7474009e90 tx=0x7f7474009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7268863f8eeb0a74 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:27.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f74837fe700 1 -- 172.21.15.143:0/1983290436 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747400e070 con 0x7f748406e930 2023-09-29T23:30:27.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f74837fe700 1 -- 172.21.15.143:0/1983290436 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747400bc80 con 0x7f748406e930 2023-09-29T23:30:27.591 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f74837fe700 1 -- 172.21.15.143:0/1983290436 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474012430 con 0x7f748406e930 2023-09-29T23:30:27.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7489c71700 1 -- 172.21.15.143:0/1983290436 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 msgr2=0x7f748406c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:27.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7489c71700 1 --2- 172.21.15.143:0/1983290436 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 0x7f748406c7d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7474009e90 tx=0x7f7474009780 comp rx=0 tx=0).stop 2023-09-29T23:30:27.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7489c71700 1 -- 172.21.15.143:0/1983290436 shutdown_connections 2023-09-29T23:30:27.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7489c71700 1 --2- 172.21.15.143:0/1983290436 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 0x7f748406c7d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:27.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7489c71700 1 -- 172.21.15.143:0/1983290436 >> 172.21.15.143:0/1983290436 conn(0x7f748406f200 msgr2=0x7f7484069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:27.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7489c71700 1 -- 172.21.15.143:0/1983290436 shutdown_connections 2023-09-29T23:30:27.592 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7489c71700 1 -- 172.21.15.143:0/1983290436 wait complete. 2023-09-29T23:30:27.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7489c71700 1 Processor -- start 2023-09-29T23:30:27.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.587+0000 7f7489c71700 1 -- start start 2023-09-29T23:30:27.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7489c71700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7484066ec0 0x7f74840a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:27.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7489c71700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 0x7f74840a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:27.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7489c71700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f748413c010 0x7f7484153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:27.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7489c71700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748409e220 con 0x7f748413c010 2023-09-29T23:30:27.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7489c71700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748409b690 con 0x7f7484066ec0 2023-09-29T23:30:27.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7489c71700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7484150c70 con 0x7f748406e930 2023-09-29T23:30:27.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7483fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 0x7f74840a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:27.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7483fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 0x7f74840a6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43968/0 (socket says 172.21.15.143:43968) 2023-09-29T23:30:27.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7483fff700 1 -- 172.21.15.143:0/2934484860 learned_addr learned my addr 172.21.15.143:0/2934484860 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:27.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7488c6f700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7484066ec0 0x7f74840a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:27.595 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7489470700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f748413c010 0x7f7484153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:27.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7483fff700 1 -- 172.21.15.143:0/2934484860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7484066ec0 msgr2=0x7f74840a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:27.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7483fff700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7484066ec0 0x7f74840a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:27.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7483fff700 1 -- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f748413c010 msgr2=0x7f7484153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:27.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7483fff700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f748413c010 0x7f7484153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:27.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7483fff700 1 -- 172.21.15.143:0/2934484860 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7474009a20 con 0x7f748406e930 2023-09-29T23:30:27.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7483fff700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 0x7f74840a6820 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7474013000 tx=0x7f7474019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:27.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7480ff9700 1 -- 172.21.15.143:0/2934484860 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474012ab0 con 0x7f748406e930 2023-09-29T23:30:27.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7480ff9700 1 -- 172.21.15.143:0/2934484860 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7474021070 con 0x7f748406e930 2023-09-29T23:30:27.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7489c71700 1 -- 172.21.15.143:0/2934484860 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7484150f30 con 0x7f748406e930 2023-09-29T23:30:27.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7480ff9700 1 -- 172.21.15.143:0/2934484860 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7474012ea0 con 0x7f748406e930 2023-09-29T23:30:27.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7489c71700 1 -- 172.21.15.143:0/2934484860 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74840a4700 con 0x7f748406e930 2023-09-29T23:30:27.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.591+0000 7f7489c71700 1 -- 172.21.15.143:0/2934484860 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748413fec0 con 0x7f748406e930 2023-09-29T23:30:27.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.595+0000 7f7480ff9700 1 -- 172.21.15.143:0/2934484860 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f7474023030 con 0x7f748406e930 2023-09-29T23:30:27.599 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.595+0000 7f7480ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:27.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.595+0000 7f7480ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:27.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.595+0000 7f7480ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:27.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.595+0000 7f7480ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:27.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.595+0000 7f7480ff9700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f747008e830 0x7f7470090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:27.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.595+0000 7f7480ff9700 1 -- 172.21.15.143:0/2934484860 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f74740b6c30 con 0x7f748406e930 2023-09-29T23:30:27.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.595+0000 7f7488c6f700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f747008e830 0x7f7470090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:27.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.599+0000 7f7488c6f700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f747008e830 0x7f7470090ce0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7478007fd0 tx=0x7f7478009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:27.604 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.599+0000 7f7480ff9700 1 -- 172.21.15.143:0/2934484860 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7474087550 con 0x7f748406e930 2023-09-29T23:30:27.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.795+0000 7f7489c71700 1 -- 172.21.15.143:0/2934484860 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f748413fec0 con 0x7f748406e930 2023-09-29T23:30:27.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.795+0000 7f7480ff9700 1 -- 172.21.15.143:0/2934484860 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f748413fec0 con 0x7f748406e930 2023-09-29T23:30:27.799 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:27.800 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:27.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 10 mgrc shutdown 2023-09-29T23:30:27.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 -- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f747008e830 msgr2=0x7f7470090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:27.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f747008e830 0x7f7470090ce0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7478007fd0 tx=0x7f7478009040 comp rx=0 tx=0).stop 2023-09-29T23:30:27.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 -- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 msgr2=0x7f74840a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:27.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 0x7f74840a6820 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7474013000 tx=0x7f7474019000 comp rx=0 tx=0).stop 2023-09-29T23:30:27.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 -- 172.21.15.143:0/2934484860 shutdown_connections 2023-09-29T23:30:27.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f747008e830 0x7f7470090ce0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:27.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f748413c010 0x7f7484153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:27.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f748406e930 0x7f74840a6820 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:27.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 --2- 172.21.15.143:0/2934484860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7484066ec0 0x7f74840a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:27.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 -- 172.21.15.143:0/2934484860 >> 172.21.15.143:0/2934484860 conn(0x7f748406f200 msgr2=0x7f748406fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:27.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 -- 172.21.15.143:0/2934484860 shutdown_connections 2023-09-29T23:30:27.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:27.799+0000 7f7489c71700 1 -- 172.21.15.143:0/2934484860 wait complete. 2023-09-29T23:30:30.818 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:30.930 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bcffff700 1 Processor -- start 2023-09-29T23:30:30.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bcffff700 1 -- start start 2023-09-29T23:30:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bcffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5bd0066ec0 0x7f5bd00aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bcffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd013a300 0x7f5bd013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bcffff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd00a19f0 con 0x7f5bd00a0d60 2023-09-29T23:30:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bcffff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd0067290 con 0x7f5bd0066ec0 2023-09-29T23:30:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bcffff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd009f560 con 0x7f5bd013a300 2023-09-29T23:30:30.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bce7fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd013a300 0x7f5bd013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bce7fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd013a300 0x7f5bd013a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43972/0 (socket says 172.21.15.143:43972) 2023-09-29T23:30:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bce7fc700 1 -- 172.21.15.143:0/2859356776 learned_addr learned my addr 172.21.15.143:0/2859356776 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bceffd700 1 --2- 172.21.15.143:0/2859356776 >> v2:172.21.15.156:3300/0 conn(0x7f5bd0066ec0 0x7f5bd00aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bce7fc700 1 -- 172.21.15.143:0/2859356776 >> v2:172.21.15.156:3300/0 conn(0x7f5bd0066ec0 msgr2=0x7f5bd00aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bce7fc700 1 --2- 172.21.15.143:0/2859356776 >> v2:172.21.15.156:3300/0 conn(0x7f5bd0066ec0 0x7f5bd00aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bce7fc700 1 -- 172.21.15.143:0/2859356776 >> v1:172.21.15.156:6789/0 conn(0x7f5bd00a0d60 legacy=0x7f5bd00a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bce7fc700 1 -- 172.21.15.143:0/2859356776 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd00a20b0 con 0x7f5bd013a300 2023-09-29T23:30:30.933 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bce7fc700 1 --2- 172.21.15.143:0/2859356776 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd013a300 0x7f5bd013a730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009f20 tx=0x7f5bc0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ab776aafac321ab server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:30.934 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.927+0000 7f5bcdffb700 1 -- 172.21.15.143:0/2859356776 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc000e070 con 0x7f5bd013a300 2023-09-29T23:30:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcdffb700 1 -- 172.21.15.143:0/2859356776 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc000bb10 con 0x7f5bd013a300 2023-09-29T23:30:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcdffb700 1 -- 172.21.15.143:0/2859356776 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00092e0 con 0x7f5bd013a300 2023-09-29T23:30:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcffff700 1 -- 172.21.15.143:0/2859356776 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd013a300 msgr2=0x7f5bd013a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcffff700 1 --2- 172.21.15.143:0/2859356776 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd013a300 0x7f5bd013a730 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009f20 tx=0x7f5bc0009840 comp rx=0 tx=0).stop 2023-09-29T23:30:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcffff700 1 -- 172.21.15.143:0/2859356776 shutdown_connections 2023-09-29T23:30:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcffff700 1 --2- 172.21.15.143:0/2859356776 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd013a300 0x7f5bd013a730 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcffff700 1 --2- 172.21.15.143:0/2859356776 >> v2:172.21.15.156:3300/0 conn(0x7f5bd0066ec0 0x7f5bd00aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcffff700 1 -- 172.21.15.143:0/2859356776 >> 172.21.15.143:0/2859356776 conn(0x7f5bd006f200 msgr2=0x7f5bd0069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcffff700 1 -- 172.21.15.143:0/2859356776 shutdown_connections 2023-09-29T23:30:30.935 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcffff700 1 -- 172.21.15.143:0/2859356776 wait complete. 2023-09-29T23:30:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcffff700 1 Processor -- start 2023-09-29T23:30:30.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.931+0000 7f5bcffff700 1 -- start start 2023-09-29T23:30:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bcffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd0066ec0 0x7f5bd00962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:30.937 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bcffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd00a0d60 0x7f5bd00976c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bcffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bd013a300 0x7f5bd0097c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bcffff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd006e300 con 0x7f5bd00a0d60 2023-09-29T23:30:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bcffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd006d020 con 0x7f5bd013a300 2023-09-29T23:30:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bcffff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd00971c0 con 0x7f5bd0066ec0 2023-09-29T23:30:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bceffd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd0066ec0 0x7f5bd00962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bceffd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd0066ec0 0x7f5bd00962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43974/0 (socket says 172.21.15.143:43974) 2023-09-29T23:30:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bceffd700 1 -- 172.21.15.143:0/2099401848 learned_addr learned my addr 172.21.15.143:0/2099401848 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bcf7fe700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bd013a300 0x7f5bd0097c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:30.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bce7fc700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd00a0d60 0x7f5bd00976c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:30.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bceffd700 1 -- 172.21.15.143:0/2099401848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bd013a300 msgr2=0x7f5bd0097c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:30.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bceffd700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bd013a300 0x7f5bd0097c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:30.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bceffd700 1 -- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd00a0d60 msgr2=0x7f5bd00976c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:30.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bceffd700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd00a0d60 0x7f5bd00976c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:30.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bceffd700 1 -- 172.21.15.143:0/2099401848 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc0009ae0 con 0x7f5bd0066ec0 2023-09-29T23:30:30.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bce7fc700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd00a0d60 0x7f5bd00976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:30:30.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bceffd700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd0066ec0 0x7f5bd00962b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400d890 tx=0x7f5bc4008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bb6ffd700 1 -- 172.21.15.143:0/2099401848 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40125f0 con 0x7f5bd0066ec0 2023-09-29T23:30:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bb6ffd700 1 -- 172.21.15.143:0/2099401848 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4012750 con 0x7f5bd0066ec0 2023-09-29T23:30:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bcffff700 1 -- 172.21.15.143:0/2099401848 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd0154120 con 0x7f5bd0066ec0 2023-09-29T23:30:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bb6ffd700 1 -- 172.21.15.143:0/2099401848 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4012b10 con 0x7f5bd0066ec0 2023-09-29T23:30:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bcffff700 1 -- 172.21.15.143:0/2099401848 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd00967f0 con 0x7f5bd0066ec0 2023-09-29T23:30:30.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.935+0000 7f5bcffff700 1 -- 172.21.15.143:0/2099401848 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd005ecb0 con 0x7f5bd0066ec0 2023-09-29T23:30:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.939+0000 7f5bb6ffd700 1 -- 172.21.15.143:0/2099401848 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f5bc401e460 con 0x7f5bd0066ec0 2023-09-29T23:30:30.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.939+0000 7f5bb6ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.939+0000 7f5bb6ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.939+0000 7f5bb6ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.939+0000 7f5bb6ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:30.942 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.939+0000 7f5bb6ffd700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5bbc08e860 0x7f5bbc090d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.939+0000 7f5bb6ffd700 1 -- 172.21.15.143:0/2099401848 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f5bc40b59b0 con 0x7f5bd0066ec0 2023-09-29T23:30:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.939+0000 7f5bce7fc700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5bbc08e860 0x7f5bbc090d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:30.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.939+0000 7f5bce7fc700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5bbc08e860 0x7f5bbc090d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009880 tx=0x7f5bc0018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:30.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:30.947+0000 7f5bb6ffd700 1 -- 172.21.15.143:0/2099401848 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f5bc4086370 con 0x7f5bd0066ec0 2023-09-29T23:30:31.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 -- 172.21.15.143:0/2099401848 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5bd00af3a0 con 0x7f5bd0066ec0 2023-09-29T23:30:31.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bb6ffd700 1 -- 172.21.15.143:0/2099401848 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f5bc4089990 con 0x7f5bd0066ec0 2023-09-29T23:30:31.146 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:31.146 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:31.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 10 mgrc shutdown 2023-09-29T23:30:31.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 -- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5bbc08e860 msgr2=0x7f5bbc090d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:31.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5bbc08e860 0x7f5bbc090d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009880 tx=0x7f5bc0018000 comp rx=0 tx=0).stop 2023-09-29T23:30:31.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 -- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd0066ec0 msgr2=0x7f5bd00962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:31.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd0066ec0 0x7f5bd00962b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400d890 tx=0x7f5bc4008040 comp rx=0 tx=0).stop 2023-09-29T23:30:31.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 -- 172.21.15.143:0/2099401848 shutdown_connections 2023-09-29T23:30:31.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f5bbc08e860 0x7f5bbc090d10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:31.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bd013a300 0x7f5bd0097c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:31.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5bd00a0d60 0x7f5bd00976c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:31.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 --2- 172.21.15.143:0/2099401848 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5bd0066ec0 0x7f5bd00962b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:31.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 -- 172.21.15.143:0/2099401848 >> 172.21.15.143:0/2099401848 conn(0x7f5bd006f200 msgr2=0x7f5bd01387d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:31.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 -- 172.21.15.143:0/2099401848 shutdown_connections 2023-09-29T23:30:31.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:31.143+0000 7f5bcffff700 1 -- 172.21.15.143:0/2099401848 wait complete. 2023-09-29T23:30:34.161 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:34.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.271+0000 7fcba759e700 1 Processor -- start 2023-09-29T23:30:34.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.271+0000 7fcba759e700 1 -- start start 2023-09-29T23:30:34.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.271+0000 7fcba759e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcba813a300 0x7fcba813a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:34.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.271+0000 7fcba759e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcba80a0d60 0x7fcba80a1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:34.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.271+0000 7fcba759e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba80a19f0 con 0x7fcba8066ec0 2023-09-29T23:30:34.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.271+0000 7fcba759e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba8067290 con 0x7fcba80a0d60 2023-09-29T23:30:34.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.271+0000 7fcba759e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba809f560 con 0x7fcba813a300 2023-09-29T23:30:34.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.271+0000 7fcba6d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcba80a0d60 0x7fcba80a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:34.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.271+0000 7fcba659c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fcba8066ec0 0x7fcba80aee60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44106/0 (socket says 172.21.15.143:44106) 2023-09-29T23:30:34.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.271+0000 7fcba659c700 1 -- 172.21.15.143:0/555889955 learned_addr learned my addr 172.21.15.143:0/555889955 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:30:34.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.271+0000 7fcba5d9b700 1 --2- 172.21.15.143:0/555889955 >> v2:172.21.15.156:3300/0 conn(0x7fcba813a300 0x7fcba813a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:34.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba559a700 1 -- 172.21.15.143:0/555889955 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397771186 0 0) 0x7fcba80a19f0 con 0x7fcba8066ec0 2023-09-29T23:30:34.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba559a700 1 -- 172.21.15.143:0/555889955 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb94003570 con 0x7fcba8066ec0 2023-09-29T23:30:34.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba5d9b700 1 -- 172.21.15.143:0/555889955 >> v2:172.21.15.143:3300/0 conn(0x7fcba80a0d60 msgr2=0x7fcba80a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:34.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba5d9b700 1 --2- 172.21.15.143:0/555889955 >> v2:172.21.15.143:3300/0 conn(0x7fcba80a0d60 0x7fcba80a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:34.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba5d9b700 1 -- 172.21.15.143:0/555889955 >> v1:172.21.15.143:6789/0 conn(0x7fcba8066ec0 legacy=0x7fcba80aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:34.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba5d9b700 1 -- 172.21.15.143:0/555889955 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba80a20b0 con 0x7fcba813a300 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba5d9b700 1 --2- 172.21.15.143:0/555889955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcba813a300 0x7fcba813a760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcba8068010 tx=0x7fcb900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1b3a3b2eab50046 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba559a700 1 -- 172.21.15.143:0/555889955 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90011630 con 0x7fcba813a300 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba559a700 1 -- 172.21.15.143:0/555889955 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb900117b0 con 0x7fcba813a300 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba559a700 1 -- 172.21.15.143:0/555889955 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90011b70 con 0x7fcba813a300 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 -- 172.21.15.143:0/555889955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcba813a300 msgr2=0x7fcba813a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 --2- 172.21.15.143:0/555889955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcba813a300 0x7fcba813a760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcba8068010 tx=0x7fcb900091a0 comp rx=0 tx=0).stop 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 -- 172.21.15.143:0/555889955 shutdown_connections 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 --2- 172.21.15.143:0/555889955 >> v2:172.21.15.143:3300/0 conn(0x7fcba80a0d60 0x7fcba80a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 --2- 172.21.15.143:0/555889955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcba813a300 0x7fcba813a760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 -- 172.21.15.143:0/555889955 >> 172.21.15.143:0/555889955 conn(0x7fcba806f200 msgr2=0x7fcba8069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 -- 172.21.15.143:0/555889955 shutdown_connections 2023-09-29T23:30:34.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 -- 172.21.15.143:0/555889955 wait complete. 2023-09-29T23:30:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 Processor -- start 2023-09-29T23:30:34.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 -- start start 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcba8066ec0 0x7fcba8138c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcba80a0d60 0x7fcba8099130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcba813a300 0x7fcba8099670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba8098520 con 0x7fcba80a0d60 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba809ef20 con 0x7fcba813a300 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba759e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba8098c10 con 0x7fcba8066ec0 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba659c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcba8066ec0 0x7fcba8138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.275+0000 7fcba659c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcba8066ec0 0x7fcba8138c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:37060/0 (socket says 172.21.15.143:37060) 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba659c700 1 -- 172.21.15.143:0/3811205613 learned_addr learned my addr 172.21.15.143:0/3811205613 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba5d9b700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcba80a0d60 0x7fcba8099130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba6d9d700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcba813a300 0x7fcba8099670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:34.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba659c700 1 -- 172.21.15.143:0/3811205613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcba813a300 msgr2=0x7fcba8099670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:34.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba659c700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcba813a300 0x7fcba8099670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:34.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba659c700 1 -- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcba80a0d60 msgr2=0x7fcba8099130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:34.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba659c700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcba80a0d60 0x7fcba8099130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:34.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba659c700 1 -- 172.21.15.143:0/3811205613 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb90009420 con 0x7fcba8066ec0 2023-09-29T23:30:34.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba659c700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcba8066ec0 0x7fcba8138c10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c7c0 tx=0x7fcba0007e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:34.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcb9e7fc700 1 -- 172.21.15.143:0/3811205613 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0005cd0 con 0x7fcba8066ec0 2023-09-29T23:30:34.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcb9e7fc700 1 -- 172.21.15.143:0/3811205613 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba0005e30 con 0x7fcba8066ec0 2023-09-29T23:30:34.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba759e700 1 -- 172.21.15.143:0/3811205613 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba806e4d0 con 0x7fcba8066ec0 2023-09-29T23:30:34.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcb9e7fc700 1 -- 172.21.15.143:0/3811205613 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0007320 con 0x7fcba8066ec0 2023-09-29T23:30:34.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba759e700 1 -- 172.21.15.143:0/3811205613 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba8095b70 con 0x7fcba8066ec0 2023-09-29T23:30:34.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.279+0000 7fcba759e700 1 -- 172.21.15.143:0/3811205613 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba805ecb0 con 0x7fcba8066ec0 2023-09-29T23:30:34.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.283+0000 7fcb9e7fc700 1 -- 172.21.15.143:0/3811205613 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fcba001b3b0 con 0x7fcba8066ec0 2023-09-29T23:30:34.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.283+0000 7fcb9e7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:34.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.283+0000 7fcb9e7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:34.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.283+0000 7fcb9e7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:34.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.283+0000 7fcb9e7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:34.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.283+0000 7fcb9e7fc700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcb8808e8b0 0x7fcb88090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:34.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.283+0000 7fcba5d9b700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcb8808e8b0 0x7fcb88090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:34.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.283+0000 7fcb9e7fc700 1 -- 172.21.15.143:0/3811205613 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fcba00b3ea0 con 0x7fcba8066ec0 2023-09-29T23:30:34.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.283+0000 7fcba5d9b700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcb8808e8b0 0x7fcb88090d60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcba8068010 tx=0x7fcb90017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:34.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.291+0000 7fcb9e7fc700 1 -- 172.21.15.143:0/3811205613 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcba00848f0 con 0x7fcba8066ec0 2023-09-29T23:30:34.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.487+0000 7fcba759e700 1 -- 172.21.15.143:0/3811205613 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcba80b0d00 con 0x7fcba8066ec0 2023-09-29T23:30:34.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcb9e7fc700 1 -- 172.21.15.143:0/3811205613 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fcba0017020 con 0x7fcba8066ec0 2023-09-29T23:30:34.493 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:34.493 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:34.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 10 mgrc shutdown 2023-09-29T23:30:34.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 -- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcb8808e8b0 msgr2=0x7fcb88090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcb8808e8b0 0x7fcb88090d60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcba8068010 tx=0x7fcb90017000 comp rx=0 tx=0).stop 2023-09-29T23:30:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 -- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcba8066ec0 msgr2=0x7fcba8138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcba8066ec0 0x7fcba8138c10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c7c0 tx=0x7fcba0007e00 comp rx=0 tx=0).stop 2023-09-29T23:30:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 -- 172.21.15.143:0/3811205613 shutdown_connections 2023-09-29T23:30:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcb8808e8b0 0x7fcb88090d60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcba813a300 0x7fcba8099670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcba80a0d60 0x7fcba8099130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 --2- 172.21.15.143:0/3811205613 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcba8066ec0 0x7fcba8138c10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 -- 172.21.15.143:0/3811205613 >> 172.21.15.143:0/3811205613 conn(0x7fcba806f200 msgr2=0x7fcba80a3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 -- 172.21.15.143:0/3811205613 shutdown_connections 2023-09-29T23:30:34.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:34.491+0000 7fcba759e700 1 -- 172.21.15.143:0/3811205613 wait complete. 2023-09-29T23:30:37.508 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:37.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481ffff700 1 Processor -- start 2023-09-29T23:30:37.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481ffff700 1 -- start start 2023-09-29T23:30:37.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481ffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4820141aa0 0x7f482009efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:37.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481ffff700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4820142020 con 0x7f482006ee30 2023-09-29T23:30:37.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481ffff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482009e200 con 0x7f482006f200 2023-09-29T23:30:37.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481ffff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48201422e0 con 0x7f4820141aa0 2023-09-29T23:30:37.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4820141aa0 0x7f482009efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:37.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481effd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f482006ee30 0x7f482009d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44110/0 (socket says 172.21.15.143:44110) 2023-09-29T23:30:37.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481effd700 1 -- 172.21.15.143:0/1394340063 learned_addr learned my addr 172.21.15.143:0/1394340063 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:30:37.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481dffb700 1 -- 172.21.15.143:0/1394340063 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532736231 0 0) 0x7f482009e200 con 0x7f482006f200 2023-09-29T23:30:37.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481f7fe700 1 -- 172.21.15.143:0/1394340063 >> v1:172.21.15.156:6789/0 conn(0x7f482006f200 legacy=0x7f4820141390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:37.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481f7fe700 1 -- 172.21.15.143:0/1394340063 >> v1:172.21.15.143:6789/0 conn(0x7f482006ee30 legacy=0x7f482009d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:37.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.619+0000 7f481f7fe700 1 -- 172.21.15.143:0/1394340063 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482009da60 con 0x7f4820141aa0 2023-09-29T23:30:37.625 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.623+0000 7f481f7fe700 1 --2- 172.21.15.143:0/1394340063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4820141aa0 0x7f482009efe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4818009e60 tx=0x7f4818009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26871d1e472d56e server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:37.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.623+0000 7f481dffb700 1 -- 172.21.15.143:0/1394340063 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800e070 con 0x7f4820141aa0 2023-09-29T23:30:37.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.623+0000 7f481dffb700 1 -- 172.21.15.143:0/1394340063 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f481800bc30 con 0x7f4820141aa0 2023-09-29T23:30:37.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.623+0000 7f481dffb700 1 -- 172.21.15.143:0/1394340063 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818012430 con 0x7f4820141aa0 2023-09-29T23:30:37.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.623+0000 7f481ffff700 1 -- 172.21.15.143:0/1394340063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4820141aa0 msgr2=0x7f482009efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:37.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.623+0000 7f481ffff700 1 --2- 172.21.15.143:0/1394340063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4820141aa0 0x7f482009efe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4818009e60 tx=0x7f4818009780 comp rx=0 tx=0).stop 2023-09-29T23:30:37.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.623+0000 7f481ffff700 1 -- 172.21.15.143:0/1394340063 shutdown_connections 2023-09-29T23:30:37.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.623+0000 7f481ffff700 1 --2- 172.21.15.143:0/1394340063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4820141aa0 0x7f482009efe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:37.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.623+0000 7f481ffff700 1 -- 172.21.15.143:0/1394340063 >> 172.21.15.143:0/1394340063 conn(0x7f48200a59b0 msgr2=0x7f482006a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:37.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.623+0000 7f481ffff700 1 -- 172.21.15.143:0/1394340063 shutdown_connections 2023-09-29T23:30:37.627 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.623+0000 7f481ffff700 1 -- 172.21.15.143:0/1394340063 wait complete. 2023-09-29T23:30:37.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481ffff700 1 Processor -- start 2023-09-29T23:30:37.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481ffff700 1 -- start start 2023-09-29T23:30:37.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482006ee30 0x7f48200a4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:37.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481ffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f482006f200 0x7f48201532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:37.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4820141aa0 0x7f4820153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:37.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481ffff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4820153d60 con 0x7f482006ee30 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481ffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4820153f50 con 0x7f4820141aa0 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481ffff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48200a41a0 con 0x7f482006f200 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482006ee30 0x7f48200a4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481e7fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f482006f200 0x7f48201532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482006ee30 0x7f48200a4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:34518/0 (socket says 172.21.15.143:34518) 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481effd700 1 -- 172.21.15.143:0/2137140423 learned_addr learned my addr 172.21.15.143:0/2137140423 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481f7fe700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4820141aa0 0x7f4820153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481effd700 1 -- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f482006f200 msgr2=0x7f48201532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481effd700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f482006f200 0x7f48201532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481effd700 1 -- 172.21.15.143:0/2137140423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4820141aa0 msgr2=0x7f4820153820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481effd700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4820141aa0 0x7f4820153820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:37.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481effd700 1 -- 172.21.15.143:0/2137140423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818009a20 con 0x7f482006ee30 2023-09-29T23:30:37.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481e7fc700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f482006f200 0x7f48201532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:30:37.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481effd700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482006ee30 0x7f48200a4b40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f481000c7c0 tx=0x7f4810007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:37.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f4806ffd700 1 -- 172.21.15.143:0/2137140423 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810005b30 con 0x7f482006ee30 2023-09-29T23:30:37.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f4806ffd700 1 -- 172.21.15.143:0/2137140423 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4810005c90 con 0x7f482006ee30 2023-09-29T23:30:37.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481ffff700 1 -- 172.21.15.143:0/2137140423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482013f610 con 0x7f482006ee30 2023-09-29T23:30:37.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f4806ffd700 1 -- 172.21.15.143:0/2137140423 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4810004a80 con 0x7f482006ee30 2023-09-29T23:30:37.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.627+0000 7f481ffff700 1 -- 172.21.15.143:0/2137140423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4820141000 con 0x7f482006ee30 2023-09-29T23:30:37.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.631+0000 7f481ffff700 1 -- 172.21.15.143:0/2137140423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4820141220 con 0x7f482006ee30 2023-09-29T23:30:37.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.631+0000 7f4806ffd700 1 -- 172.21.15.143:0/2137140423 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f48100070d0 con 0x7f482006ee30 2023-09-29T23:30:37.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.631+0000 7f4806ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:37.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.635+0000 7f4806ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:37.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.635+0000 7f4806ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:37.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.635+0000 7f4806ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:37.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.635+0000 7f4806ffd700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f480c08e8b0 0x7f480c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:37.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.635+0000 7f481e7fc700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f480c08e8b0 0x7f480c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:37.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.635+0000 7f4806ffd700 1 -- 172.21.15.143:0/2137140423 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f48100b47c0 con 0x7f482006ee30 2023-09-29T23:30:37.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.635+0000 7f481e7fc700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f480c08e8b0 0x7f480c090d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f48140011e0 tx=0x7f4814009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:37.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.635+0000 7f4806ffd700 1 -- 172.21.15.143:0/2137140423 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4810085180 con 0x7f482006ee30 2023-09-29T23:30:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 -- 172.21.15.143:0/2137140423 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f482005ecb0 con 0x7f482006ee30 2023-09-29T23:30:37.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f4806ffd700 1 -- 172.21.15.143:0/2137140423 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f4810021070 con 0x7f482006ee30 2023-09-29T23:30:37.834 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:37.834 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:37.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 10 mgrc shutdown 2023-09-29T23:30:37.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 -- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f480c08e8b0 msgr2=0x7f480c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:37.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f480c08e8b0 0x7f480c090d60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f48140011e0 tx=0x7f4814009040 comp rx=0 tx=0).stop 2023-09-29T23:30:37.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 -- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482006ee30 msgr2=0x7f48200a4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:37.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482006ee30 0x7f48200a4b40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f481000c7c0 tx=0x7f4810007f70 comp rx=0 tx=0).stop 2023-09-29T23:30:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 -- 172.21.15.143:0/2137140423 shutdown_connections 2023-09-29T23:30:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f480c08e8b0 0x7f480c090d60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4820141aa0 0x7f4820153820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f482006f200 0x7f48201532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 --2- 172.21.15.143:0/2137140423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f482006ee30 0x7f48200a4b40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 -- 172.21.15.143:0/2137140423 >> 172.21.15.143:0/2137140423 conn(0x7f48200a59b0 msgr2=0x7f48200673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 -- 172.21.15.143:0/2137140423 shutdown_connections 2023-09-29T23:30:37.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:37.831+0000 7f481ffff700 1 -- 172.21.15.143:0/2137140423 wait complete. 2023-09-29T23:30:40.849 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.963+0000 7f278752c700 1 Processor -- start 2023-09-29T23:30:40.967 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.963+0000 7f278752c700 1 -- start start 2023-09-29T23:30:40.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.963+0000 7f278752c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f278009bcd0 0x7f2780147c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:40.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.963+0000 7f278752c700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27800a2160 con 0x7f27800a0550 2023-09-29T23:30:40.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.963+0000 7f278752c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27800a3c60 con 0x7f27800a0180 2023-09-29T23:30:40.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.963+0000 7f278752c700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27800a3f20 con 0x7f278009bcd0 2023-09-29T23:30:40.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2785d29700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f27800a0550 0x7f27800a1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:44122/0 (socket says 172.21.15.143:44122) 2023-09-29T23:30:40.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2785d29700 1 -- 172.21.15.143:0/1973787879 learned_addr learned my addr 172.21.15.143:0/1973787879 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:30:40.969 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2786d2b700 1 --2- 172.21.15.143:0/1973787879 >> v2:172.21.15.156:3300/0 conn(0x7f278009bcd0 0x7f2780147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:40.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2785528700 1 -- 172.21.15.143:0/1973787879 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365540064 0 0) 0x7f27800a2160 con 0x7f27800a0550 2023-09-29T23:30:40.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2785528700 1 -- 172.21.15.143:0/1973787879 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f276c003570 con 0x7f27800a0550 2023-09-29T23:30:40.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2786d2b700 1 -- 172.21.15.143:0/1973787879 >> v1:172.21.15.156:6789/0 conn(0x7f27800a0180 legacy=0x7f27800a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:40.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2786d2b700 1 -- 172.21.15.143:0/1973787879 >> v1:172.21.15.143:6789/0 conn(0x7f27800a0550 legacy=0x7f27800a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:40.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2786d2b700 1 -- 172.21.15.143:0/1973787879 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27800a3890 con 0x7f278009bcd0 2023-09-29T23:30:40.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2786d2b700 1 --2- 172.21.15.143:0/1973787879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f278009bcd0 0x7f2780147c00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f277800af70 tx=0x7f277800a890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71057d6e7ac7899e server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:40.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2785528700 1 -- 172.21.15.143:0/1973787879 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277800f070 con 0x7f278009bcd0 2023-09-29T23:30:40.970 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2785528700 1 -- 172.21.15.143:0/1973787879 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277800a2e0 con 0x7f278009bcd0 2023-09-29T23:30:40.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f2785528700 1 -- 172.21.15.143:0/1973787879 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277800cc30 con 0x7f278009bcd0 2023-09-29T23:30:40.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f278752c700 1 -- 172.21.15.143:0/1973787879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f278009bcd0 msgr2=0x7f2780147c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:40.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f278752c700 1 --2- 172.21.15.143:0/1973787879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f278009bcd0 0x7f2780147c00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f277800af70 tx=0x7f277800a890 comp rx=0 tx=0).stop 2023-09-29T23:30:40.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f278752c700 1 -- 172.21.15.143:0/1973787879 shutdown_connections 2023-09-29T23:30:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f278752c700 1 --2- 172.21.15.143:0/1973787879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f278009bcd0 0x7f2780147c00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f278752c700 1 -- 172.21.15.143:0/1973787879 >> 172.21.15.143:0/1973787879 conn(0x7f278006a470 msgr2=0x7f278006a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f278752c700 1 -- 172.21.15.143:0/1973787879 shutdown_connections 2023-09-29T23:30:40.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.967+0000 7f278752c700 1 -- 172.21.15.143:0/1973787879 wait complete. 2023-09-29T23:30:40.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278752c700 1 Processor -- start 2023-09-29T23:30:40.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278752c700 1 -- start start 2023-09-29T23:30:40.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278752c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f278009bcd0 0x7f2780142820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:40.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278752c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27800a0180 0x7f27800961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:40.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278752c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27800a0550 0x7f2780139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:40.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278752c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278006b7e0 con 0x7f27800a0550 2023-09-29T23:30:40.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278752c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278006baa0 con 0x7f27800a0180 2023-09-29T23:30:40.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278752c700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2780096720 con 0x7f278009bcd0 2023-09-29T23:30:40.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278652a700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f278009bcd0 0x7f2780142820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:40.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278652a700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f278009bcd0 0x7f2780142820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:37082/0 (socket says 172.21.15.143:37082) 2023-09-29T23:30:40.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278652a700 1 -- 172.21.15.143:0/1600381304 learned_addr learned my addr 172.21.15.143:0/1600381304 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:40.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f2786d2b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27800a0550 0x7f2780139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:40.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f2785d29700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27800a0180 0x7f27800961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278652a700 1 -- 172.21.15.143:0/1600381304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27800a0180 msgr2=0x7f27800961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278652a700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27800a0180 0x7f27800961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278652a700 1 -- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27800a0550 msgr2=0x7f2780139560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278652a700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27800a0550 0x7f2780139560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278652a700 1 -- 172.21.15.143:0/1600381304 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277800ab30 con 0x7f278009bcd0 2023-09-29T23:30:40.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278652a700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f278009bcd0 0x7f2780142820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f277c00c590 tx=0x7f277c00c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:40.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f27767fc700 1 -- 172.21.15.143:0/1600381304 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c011590 con 0x7f278009bcd0 2023-09-29T23:30:40.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f27767fc700 1 -- 172.21.15.143:0/1600381304 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c011710 con 0x7f278009bcd0 2023-09-29T23:30:40.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278752c700 1 -- 172.21.15.143:0/1600381304 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278009b980 con 0x7f278009bcd0 2023-09-29T23:30:40.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f27767fc700 1 -- 172.21.15.143:0/1600381304 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c011ad0 con 0x7f278009bcd0 2023-09-29T23:30:40.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.971+0000 7f278752c700 1 -- 172.21.15.143:0/1600381304 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278006b560 con 0x7f278009bcd0 2023-09-29T23:30:40.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.975+0000 7f278752c700 1 -- 172.21.15.143:0/1600381304 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278005ecb0 con 0x7f278009bcd0 2023-09-29T23:30:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.975+0000 7f27767fc700 1 -- 172.21.15.143:0/1600381304 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f277c01b3b0 con 0x7f278009bcd0 2023-09-29T23:30:40.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.975+0000 7f27767fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:40.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.979+0000 7f27767fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:40.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.979+0000 7f27767fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:40.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.979+0000 7f27767fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:40.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.979+0000 7f27767fc700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f276408e8b0 0x7f2764090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:40.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.979+0000 7f2785d29700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f276408e8b0 0x7f2764090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:40.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.979+0000 7f27767fc700 1 -- 172.21.15.143:0/1600381304 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f277c0b3e00 con 0x7f278009bcd0 2023-09-29T23:30:40.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.979+0000 7f2785d29700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f276408e8b0 0x7f2764090d60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2770007fd0 tx=0x7f2770009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:40.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:40.983+0000 7f27767fc700 1 -- 172.21.15.143:0/1600381304 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f277c084720 con 0x7f278009bcd0 2023-09-29T23:30:41.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.175+0000 7f278752c700 1 -- 172.21.15.143:0/1600381304 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2780096860 con 0x7f278009bcd0 2023-09-29T23:30:41.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f27767fc700 1 -- 172.21.15.143:0/1600381304 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f277c017070 con 0x7f278009bcd0 2023-09-29T23:30:41.181 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:41.182 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:41.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 10 mgrc shutdown 2023-09-29T23:30:41.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 -- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f276408e8b0 msgr2=0x7f2764090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:41.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f276408e8b0 0x7f2764090d60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2770007fd0 tx=0x7f2770009040 comp rx=0 tx=0).stop 2023-09-29T23:30:41.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 -- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f278009bcd0 msgr2=0x7f2780142820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:41.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f278009bcd0 0x7f2780142820 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f277c00c590 tx=0x7f277c00c8a0 comp rx=0 tx=0).stop 2023-09-29T23:30:41.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 -- 172.21.15.143:0/1600381304 shutdown_connections 2023-09-29T23:30:41.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f276408e8b0 0x7f2764090d60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:41.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27800a0550 0x7f2780139560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:41.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27800a0180 0x7f27800961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:41.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 --2- 172.21.15.143:0/1600381304 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f278009bcd0 0x7f2780142820 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:41.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 -- 172.21.15.143:0/1600381304 >> 172.21.15.143:0/1600381304 conn(0x7f278006a470 msgr2=0x7f278006ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:41.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 -- 172.21.15.143:0/1600381304 shutdown_connections 2023-09-29T23:30:41.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:41.179+0000 7f278752c700 1 -- 172.21.15.143:0/1600381304 wait complete. 2023-09-29T23:30:44.201 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:44.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7539522700 1 Processor -- start 2023-09-29T23:30:44.313 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7539522700 1 -- start start 2023-09-29T23:30:44.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7539522700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7534066ec0 0x7f75340a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:44.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7539522700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f753409f830 0x7f753413d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:44.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7539522700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409fc70 0x7f753413d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:44.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7539522700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753413dbc0 con 0x7f753409f830 2023-09-29T23:30:44.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7539522700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75340a16b0 con 0x7f7534066ec0 2023-09-29T23:30:44.314 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7539522700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7534067290 con 0x7f753409fc70 2023-09-29T23:30:44.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7538d21700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409fc70 0x7f753413d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:44.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f752b7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f753409f830 0x7f753413d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:44.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7538d21700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409fc70 0x7f753413d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:48034/0 (socket says 172.21.15.143:48034) 2023-09-29T23:30:44.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7538d21700 1 -- 172.21.15.143:0/3505641523 learned_addr learned my addr 172.21.15.143:0/3505641523 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:44.315 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7533fff700 1 --2- 172.21.15.143:0/3505641523 >> v2:172.21.15.156:3300/0 conn(0x7f7534066ec0 0x7f75340a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:44.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f752b7fe700 1 -- 172.21.15.143:0/3505641523 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409fc70 msgr2=0x7f753413d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:44.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f752b7fe700 1 --2- 172.21.15.143:0/3505641523 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409fc70 0x7f753413d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f752b7fe700 1 -- 172.21.15.143:0/3505641523 >> v2:172.21.15.156:3300/0 conn(0x7f7534066ec0 msgr2=0x7f75340a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:44.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f752b7fe700 1 --2- 172.21.15.143:0/3505641523 >> v2:172.21.15.156:3300/0 conn(0x7f7534066ec0 0x7f75340a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f752b7fe700 1 -- 172.21.15.143:0/3505641523 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75340a0510 con 0x7f753409f830 2023-09-29T23:30:44.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7538d21700 1 --2- 172.21.15.143:0/3505641523 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409fc70 0x7f753413d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:30:44.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7533fff700 1 --2- 172.21.15.143:0/3505641523 >> v2:172.21.15.156:3300/0 conn(0x7f7534066ec0 0x7f75340a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:30:44.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f752b7fe700 1 --2- 172.21.15.143:0/3505641523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f753409f830 0x7f753413d140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f75340a4fb0 tx=0x7f751c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea774c8c52ab871b server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:44.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f75337fe700 1 -- 172.21.15.143:0/3505641523 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c011630 con 0x7f753409f830 2023-09-29T23:30:44.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f75337fe700 1 -- 172.21.15.143:0/3505641523 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751c011790 con 0x7f753409f830 2023-09-29T23:30:44.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f75337fe700 1 -- 172.21.15.143:0/3505641523 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c011b30 con 0x7f753409f830 2023-09-29T23:30:44.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7539522700 1 -- 172.21.15.143:0/3505641523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f753409f830 msgr2=0x7f753413d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:44.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.311+0000 7f7539522700 1 --2- 172.21.15.143:0/3505641523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f753409f830 0x7f753413d140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f75340a4fb0 tx=0x7f751c0091a0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.315+0000 7f7539522700 1 -- 172.21.15.143:0/3505641523 shutdown_connections 2023-09-29T23:30:44.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.315+0000 7f7539522700 1 --2- 172.21.15.143:0/3505641523 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409fc70 0x7f753413d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.315+0000 7f7539522700 1 --2- 172.21.15.143:0/3505641523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f753409f830 0x7f753413d140 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.315+0000 7f7539522700 1 --2- 172.21.15.143:0/3505641523 >> v2:172.21.15.156:3300/0 conn(0x7f7534066ec0 0x7f75340a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.315+0000 7f7539522700 1 -- 172.21.15.143:0/3505641523 >> 172.21.15.143:0/3505641523 conn(0x7f753406f200 msgr2=0x7f7534069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:44.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.315+0000 7f7539522700 1 -- 172.21.15.143:0/3505641523 shutdown_connections 2023-09-29T23:30:44.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.315+0000 7f7539522700 1 -- 172.21.15.143:0/3505641523 wait complete. 2023-09-29T23:30:44.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.315+0000 7f7539522700 1 Processor -- start 2023-09-29T23:30:44.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.315+0000 7f7539522700 1 -- start start 2023-09-29T23:30:44.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7539522700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7534066ec0 0x7f753406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:44.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7539522700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409f830 0x7f75340aafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:44.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7539522700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753409fc70 0x7f75340ab500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:44.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7539522700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7534136510 con 0x7f7534066ec0 2023-09-29T23:30:44.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7539522700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7534140660 con 0x7f753409fc70 2023-09-29T23:30:44.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7539522700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7534140920 con 0x7f753409f830 2023-09-29T23:30:44.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f752b7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409f830 0x7f75340aafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:44.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7533fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7534066ec0 0x7f753406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:44.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f752b7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409f830 0x7f75340aafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:48038/0 (socket says 172.21.15.143:48038) 2023-09-29T23:30:44.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7533fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7534066ec0 0x7f753406c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57222/0 (socket says 172.21.15.143:57222) 2023-09-29T23:30:44.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f752b7fe700 1 -- 172.21.15.143:0/2571535936 learned_addr learned my addr 172.21.15.143:0/2571535936 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:44.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7538d21700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753409fc70 0x7f75340ab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:44.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7533fff700 1 -- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409f830 msgr2=0x7f75340aafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:44.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7533fff700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409f830 0x7f75340aafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7533fff700 1 -- 172.21.15.143:0/2571535936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753409fc70 msgr2=0x7f75340ab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7533fff700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753409fc70 0x7f75340ab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7533fff700 1 -- 172.21.15.143:0/2571535936 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7524009420 con 0x7f7534066ec0 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7538d21700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753409fc70 0x7f75340ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f752b7fe700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409f830 0x7f75340aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7533fff700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7534066ec0 0x7f753406c7c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f752400ec80 tx=0x7f75240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7530ff9700 1 -- 172.21.15.143:0/2571535936 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400c850 con 0x7f7534066ec0 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7530ff9700 1 -- 172.21.15.143:0/2571535936 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752400c9b0 con 0x7f7534066ec0 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7539522700 1 -- 172.21.15.143:0/2571535936 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c009ae0 con 0x7f7534066ec0 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7530ff9700 1 -- 172.21.15.143:0/2571535936 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752400cd50 con 0x7f7534066ec0 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7539522700 1 -- 172.21.15.143:0/2571535936 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751c009420 con 0x7f7534066ec0 2023-09-29T23:30:44.324 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.319+0000 7f7539522700 1 -- 172.21.15.143:0/2571535936 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753406cdc0 con 0x7f7534066ec0 2023-09-29T23:30:44.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.323+0000 7f7530ff9700 1 -- 172.21.15.143:0/2571535936 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f7524013680 con 0x7f7534066ec0 2023-09-29T23:30:44.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.323+0000 7f7530ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:44.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.323+0000 7f7530ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:44.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.323+0000 7f7530ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:44.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.323+0000 7f7530ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:44.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.323+0000 7f7530ff9700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751408e8b0 0x7f7514090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:44.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.323+0000 7f752b7fe700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751408e8b0 0x7f7514090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:44.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.323+0000 7f7530ff9700 1 -- 172.21.15.143:0/2571535936 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f75240b4ba0 con 0x7f7534066ec0 2023-09-29T23:30:44.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.323+0000 7f752b7fe700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751408e8b0 0x7f7514090d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f75340b1b40 tx=0x7f751c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:44.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.331+0000 7f7530ff9700 1 -- 172.21.15.143:0/2571535936 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7524085560 con 0x7f7534066ec0 2023-09-29T23:30:44.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.535+0000 7f7539522700 1 -- 172.21.15.143:0/2571535936 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f753405ecb0 con 0x7f7534066ec0 2023-09-29T23:30:44.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7530ff9700 1 -- 172.21.15.143:0/2571535936 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f7524021100 con 0x7f7534066ec0 2023-09-29T23:30:44.541 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:44.541 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:44.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 10 mgrc shutdown 2023-09-29T23:30:44.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 -- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751408e8b0 msgr2=0x7f7514090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:44.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751408e8b0 0x7f7514090d60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f75340b1b40 tx=0x7f751c017000 comp rx=0 tx=0).stop 2023-09-29T23:30:44.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 -- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7534066ec0 msgr2=0x7f753406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:44.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7534066ec0 0x7f753406c7c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f752400ec80 tx=0x7f75240091a0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 -- 172.21.15.143:0/2571535936 shutdown_connections 2023-09-29T23:30:44.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f751408e8b0 0x7f7514090d60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753409fc70 0x7f75340ab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f753409f830 0x7f75340aafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 --2- 172.21.15.143:0/2571535936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7534066ec0 0x7f753406c7c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:44.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 -- 172.21.15.143:0/2571535936 >> 172.21.15.143:0/2571535936 conn(0x7f753406f200 msgr2=0x7f75340a0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:44.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 -- 172.21.15.143:0/2571535936 shutdown_connections 2023-09-29T23:30:44.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:44.539+0000 7f7539522700 1 -- 172.21.15.143:0/2571535936 wait complete. 2023-09-29T23:30:47.555 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:47.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.662+0000 7f82affff700 1 Processor -- start 2023-09-29T23:30:47.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.662+0000 7f82affff700 1 -- start start 2023-09-29T23:30:47.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82affff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82b013c500 0x7f82b01486c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:47.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82affff700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b00a55b0 con 0x7f82b00b3820 2023-09-29T23:30:47.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82affff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b00a0020 con 0x7f82b00a0600 2023-09-29T23:30:47.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82affff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b00b3bf0 con 0x7f82b013c500 2023-09-29T23:30:47.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82af7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82b013c500 0x7f82b01486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:47.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82af7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f82b013c500 0x7f82b01486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57232/0 (socket says 172.21.15.143:57232) 2023-09-29T23:30:47.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82aeffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f82b00b3820 0x7f82b00a2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46704/0 (socket says 172.21.15.143:46704) 2023-09-29T23:30:47.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82af7fe700 1 -- 172.21.15.143:0/798519725 learned_addr learned my addr 172.21.15.143:0/798519725 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:47.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82af7fe700 1 -- 172.21.15.143:0/798519725 >> v1:172.21.15.156:6789/0 conn(0x7f82b00a0600 legacy=0x7f82b00a5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:47.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82af7fe700 1 -- 172.21.15.143:0/798519725 >> v1:172.21.15.143:6789/0 conn(0x7f82b00b3820 legacy=0x7f82b00a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:47.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82af7fe700 1 -- 172.21.15.143:0/798519725 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00a0b40 con 0x7f82b013c500 2023-09-29T23:30:47.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82af7fe700 1 --2- 172.21.15.143:0/798519725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b013c500 0x7f82b01486c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82b00a0d30 tx=0x7f82a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6264f3dd99fa83b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:47.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82ae7fc700 1 -- 172.21.15.143:0/798519725 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800e070 con 0x7f82b013c500 2023-09-29T23:30:47.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82ae7fc700 1 -- 172.21.15.143:0/798519725 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a800bae0 con 0x7f82b013c500 2023-09-29T23:30:47.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82ae7fc700 1 -- 172.21.15.143:0/798519725 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800bea0 con 0x7f82b013c500 2023-09-29T23:30:47.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82affff700 1 -- 172.21.15.143:0/798519725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b013c500 msgr2=0x7f82b01486c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:47.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82affff700 1 --2- 172.21.15.143:0/798519725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b013c500 0x7f82b01486c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f82b00a0d30 tx=0x7f82a8009840 comp rx=0 tx=0).stop 2023-09-29T23:30:47.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82affff700 1 -- 172.21.15.143:0/798519725 shutdown_connections 2023-09-29T23:30:47.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82affff700 1 --2- 172.21.15.143:0/798519725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b013c500 0x7f82b01486c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:47.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82affff700 1 -- 172.21.15.143:0/798519725 >> 172.21.15.143:0/798519725 conn(0x7f82b006ac10 msgr2=0x7f82b006afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:47.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82affff700 1 -- 172.21.15.143:0/798519725 shutdown_connections 2023-09-29T23:30:47.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.666+0000 7f82affff700 1 -- 172.21.15.143:0/798519725 wait complete. 2023-09-29T23:30:47.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82affff700 1 Processor -- start 2023-09-29T23:30:47.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82affff700 1 -- start start 2023-09-29T23:30:47.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82affff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00a0600 0x7f82b0068550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:47.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82affff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82b00b3820 0x7f82b014bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:47.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82affff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82b013c500 0x7f82b006d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:47.677 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82affff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b014c600 con 0x7f82b00a0600 2023-09-29T23:30:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82affff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b014c8c0 con 0x7f82b013c500 2023-09-29T23:30:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82affff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b014c310 con 0x7f82b00b3820 2023-09-29T23:30:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82aeffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00a0600 0x7f82b0068550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82a67fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82b00b3820 0x7f82b014bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82a67fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82b00b3820 0x7f82b014bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:48046/0 (socket says 172.21.15.143:48046) 2023-09-29T23:30:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82a67fc700 1 -- 172.21.15.143:0/2812532963 learned_addr learned my addr 172.21.15.143:0/2812532963 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82af7fe700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82b013c500 0x7f82b006d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82aeffd700 1 -- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82b00b3820 msgr2=0x7f82b014bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82aeffd700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82b00b3820 0x7f82b014bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:47.678 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82aeffd700 1 -- 172.21.15.143:0/2812532963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82b013c500 msgr2=0x7f82b006d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:47.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82aeffd700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82b013c500 0x7f82b006d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:47.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82aeffd700 1 -- 172.21.15.143:0/2812532963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a8009ae0 con 0x7f82b00a0600 2023-09-29T23:30:47.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82a67fc700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82b00b3820 0x7f82b014bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:30:47.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82aeffd700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00a0600 0x7f82b0068550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82a000c7c0 tx=0x7f82a0007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:47.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82a77fe700 1 -- 172.21.15.143:0/2812532963 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0005a70 con 0x7f82b00a0600 2023-09-29T23:30:47.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82a77fe700 1 -- 172.21.15.143:0/2812532963 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a0005bd0 con 0x7f82b00a0600 2023-09-29T23:30:47.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82affff700 1 -- 172.21.15.143:0/2812532963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b014cee0 con 0x7f82b00a0600 2023-09-29T23:30:47.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82a77fe700 1 -- 172.21.15.143:0/2812532963 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a00070d0 con 0x7f82b00a0600 2023-09-29T23:30:47.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.670+0000 7f82affff700 1 -- 172.21.15.143:0/2812532963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b013d820 con 0x7f82b00a0600 2023-09-29T23:30:47.679 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.674+0000 7f82affff700 1 -- 172.21.15.143:0/2812532963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b005ecb0 con 0x7f82b00a0600 2023-09-29T23:30:47.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.674+0000 7f82a77fe700 1 -- 172.21.15.143:0/2812532963 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f82a0007480 con 0x7f82b00a0600 2023-09-29T23:30:47.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.674+0000 7f82a77fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:47.681 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.678+0000 7f82a77fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.678+0000 7f82a77fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.678+0000 7f82a77fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.678+0000 7f82a77fe700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f829008e830 0x7f8290090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.678+0000 7f82a67fc700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f829008e830 0x7f8290090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.678+0000 7f82a77fe700 1 -- 172.21.15.143:0/2812532963 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f82a00b3d20 con 0x7f82b00a0600 2023-09-29T23:30:47.682 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.678+0000 7f82a67fc700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f829008e830 0x7f8290090ce0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f82b006e3b0 tx=0x7f8298009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:47.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.686+0000 7f82a77fe700 1 -- 172.21.15.143:0/2812532963 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f82a0084770 con 0x7f82b00a0600 2023-09-29T23:30:47.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.882+0000 7f82affff700 1 -- 172.21.15.143:0/2812532963 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f82b00a2c00 con 0x7f82b00a0600 2023-09-29T23:30:47.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82a77fe700 1 -- 172.21.15.143:0/2812532963 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f82a0017080 con 0x7f82b00a0600 2023-09-29T23:30:47.889 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:47.889 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:47.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 10 mgrc shutdown 2023-09-29T23:30:47.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 -- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f829008e830 msgr2=0x7f8290090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:47.891 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f829008e830 0x7f8290090ce0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f82b006e3b0 tx=0x7f8298009040 comp rx=0 tx=0).stop 2023-09-29T23:30:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 -- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00a0600 msgr2=0x7f82b0068550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00a0600 0x7f82b0068550 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f82a000c7c0 tx=0x7f82a0007eb0 comp rx=0 tx=0).stop 2023-09-29T23:30:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 -- 172.21.15.143:0/2812532963 shutdown_connections 2023-09-29T23:30:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f829008e830 0x7f8290090ce0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82b013c500 0x7f82b006d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f82b00b3820 0x7f82b014bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 --2- 172.21.15.143:0/2812532963 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82b00a0600 0x7f82b0068550 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 -- 172.21.15.143:0/2812532963 >> 172.21.15.143:0/2812532963 conn(0x7f82b006ac10 msgr2=0x7f82b0138650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 -- 172.21.15.143:0/2812532963 shutdown_connections 2023-09-29T23:30:47.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:47.886+0000 7f82affff700 1 -- 172.21.15.143:0/2812532963 wait complete. 2023-09-29T23:30:50.904 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:51.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 Processor -- start 2023-09-29T23:30:51.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 -- start start 2023-09-29T23:30:51.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 0x7f975c09e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:51.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f975c09db10 0x7f975c09f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:51.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f975c09df00 0x7f975c132650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:51.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975c09d020 con 0x7f975c09db10 2023-09-29T23:30:51.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f975c0a42a0 con 0x7f975c09df00 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975c0a4540 con 0x7f975c09d740 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f9759d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f975c09db10 0x7f975c09f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f9759d9b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f975c09db10 0x7f975c09f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:57258/0 (socket says 172.21.15.143:57258) 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f9759d9b700 1 -- 172.21.15.143:0/3788721241 learned_addr learned my addr 172.21.15.143:0/3788721241 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975a59c700 1 --2- 172.21.15.143:0/3788721241 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 0x7f975c09e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975ad9d700 1 --2- 172.21.15.143:0/3788721241 >> v2:172.21.15.156:3300/0 conn(0x7f975c09df00 0x7f975c132650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f9759d9b700 1 -- 172.21.15.143:0/3788721241 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 msgr2=0x7f975c09e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f9759d9b700 1 --2- 172.21.15.143:0/3788721241 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 0x7f975c09e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f9759d9b700 1 -- 172.21.15.143:0/3788721241 >> v2:172.21.15.156:3300/0 conn(0x7f975c09df00 msgr2=0x7f975c132650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f9759d9b700 1 --2- 172.21.15.143:0/3788721241 >> v2:172.21.15.156:3300/0 conn(0x7f975c09df00 0x7f975c132650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f9759d9b700 1 -- 172.21.15.143:0/3788721241 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c09eb10 con 0x7f975c09db10 2023-09-29T23:30:51.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975ad9d700 1 --2- 172.21.15.143:0/3788721241 >> v2:172.21.15.156:3300/0 conn(0x7f975c09df00 0x7f975c132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-29T23:30:51.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f9759d9b700 1 --2- 172.21.15.143:0/3788721241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f975c09db10 0x7f975c09f050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f975c140c60 tx=0x7f9754009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cf33121507b3b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:51.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975959a700 1 -- 172.21.15.143:0/3788721241 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975400e070 con 0x7f975c09db10 2023-09-29T23:30:51.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975959a700 1 -- 172.21.15.143:0/3788721241 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975400bab0 con 0x7f975c09db10 2023-09-29T23:30:51.025 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975959a700 1 -- 172.21.15.143:0/3788721241 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97540092e0 con 0x7f975c09db10 2023-09-29T23:30:51.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 -- 172.21.15.143:0/3788721241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f975c09db10 msgr2=0x7f975c09f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:51.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 --2- 172.21.15.143:0/3788721241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f975c09db10 0x7f975c09f050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f975c140c60 tx=0x7f9754009840 comp rx=0 tx=0).stop 2023-09-29T23:30:51.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 -- 172.21.15.143:0/3788721241 shutdown_connections 2023-09-29T23:30:51.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 --2- 172.21.15.143:0/3788721241 >> v2:172.21.15.156:3300/0 conn(0x7f975c09df00 0x7f975c132650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 --2- 172.21.15.143:0/3788721241 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f975c09db10 0x7f975c09f050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 --2- 172.21.15.143:0/3788721241 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 0x7f975c09e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 -- 172.21.15.143:0/3788721241 >> 172.21.15.143:0/3788721241 conn(0x7f975c0a59b0 msgr2=0x7f975c06a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:51.026 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 -- 172.21.15.143:0/3788721241 shutdown_connections 2023-09-29T23:30:51.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.018+0000 7f975b59e700 1 -- 172.21.15.143:0/3788721241 wait complete. 2023-09-29T23:30:51.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975b59e700 1 Processor -- start 2023-09-29T23:30:51.027 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975b59e700 1 -- start start 2023-09-29T23:30:51.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975b59e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 0x7f975c14ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:51.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975b59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f975c09db10 0x7f975c143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:51.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975b59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f975c09df00 0x7f975c09ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:51.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975b59e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c14f940 con 0x7f975c09db10 2023-09-29T23:30:51.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975b59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f975c1435a0 con 0x7f975c09df00 2023-09-29T23:30:51.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975b59e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975c1508e0 con 0x7f975c09d740 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975a59c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 0x7f975c14ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975a59c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 0x7f975c14ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:48054/0 (socket says 172.21.15.143:48054) 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975a59c700 1 -- 172.21.15.143:0/1506057330 learned_addr learned my addr 172.21.15.143:0/1506057330 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f9759d9b700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f975c09db10 0x7f975c143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975ad9d700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f975c09df00 0x7f975c09ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975a59c700 1 -- 172.21.15.143:0/1506057330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f975c09df00 msgr2=0x7f975c09ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975a59c700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f975c09df00 0x7f975c09ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975a59c700 1 -- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f975c09db10 msgr2=0x7f975c143f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975a59c700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f975c09db10 0x7f975c143f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975a59c700 1 -- 172.21.15.143:0/1506057330 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9754009ae0 con 0x7f975c09d740 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975ad9d700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f975c09df00 0x7f975c09ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:30:51.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975a59c700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 0x7f975c14ffb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f974400e930 tx=0x7f97440091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:51.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f97527fc700 1 -- 172.21.15.143:0/1506057330 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744013680 con 0x7f975c09d740 2023-09-29T23:30:51.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f97527fc700 1 -- 172.21.15.143:0/1506057330 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97440137e0 con 0x7f975c09d740 2023-09-29T23:30:51.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f975b59e700 1 -- 172.21.15.143:0/1506057330 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975c150b80 con 0x7f975c09d740 2023-09-29T23:30:51.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.022+0000 7f97527fc700 1 -- 172.21.15.143:0/1506057330 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9744013b80 con 0x7f975c09d740 2023-09-29T23:30:51.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.026+0000 7f975b59e700 1 -- 172.21.15.143:0/1506057330 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975c1504f0 con 0x7f975c09d740 2023-09-29T23:30:51.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.026+0000 7f97527fc700 1 -- 172.21.15.143:0/1506057330 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f974400c460 con 0x7f975c09d740 2023-09-29T23:30:51.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.026+0000 7f97527fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:51.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.026+0000 7f975b59e700 1 -- 172.21.15.143:0/1506057330 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f975c05ecb0 con 0x7f975c09d740 2023-09-29T23:30:51.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.030+0000 7f97527fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:51.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.030+0000 7f97527fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:51.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.030+0000 7f97527fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:51.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.030+0000 7f97527fc700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f973c08e8b0 0x7f973c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:51.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.030+0000 7f9759d9b700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f973c08e8b0 0x7f973c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:51.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.030+0000 7f97527fc700 1 -- 172.21.15.143:0/1506057330 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f97440b4e40 con 0x7f975c09d740 2023-09-29T23:30:51.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.030+0000 7f9759d9b700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f973c08e8b0 0x7f973c090d60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9754009880 tx=0x7f9754018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:51.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.034+0000 7f97527fc700 1 -- 172.21.15.143:0/1506057330 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9744085760 con 0x7f975c09d740 2023-09-29T23:30:51.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 1 -- 172.21.15.143:0/1506057330 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f975c09f590 con 0x7f975c09d740 2023-09-29T23:30:51.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f97527fc700 1 -- 172.21.15.143:0/1506057330 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f9744021100 con 0x7f975c09d740 2023-09-29T23:30:51.238 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:51.238 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:51.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 10 mgrc shutdown 2023-09-29T23:30:51.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 1 -- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f973c08e8b0 msgr2=0x7f973c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f973c08e8b0 0x7f973c090d60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9754009880 tx=0x7f9754018000 comp rx=0 tx=0).stop 2023-09-29T23:30:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 1 -- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 msgr2=0x7f975c14ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 0x7f975c14ffb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f974400e930 tx=0x7f97440091a0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 1 -- 172.21.15.143:0/1506057330 shutdown_connections 2023-09-29T23:30:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f973c08e8b0 0x7f973c090d60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f975c09df00 0x7f975c09ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f975c09db10 0x7f975c143f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 1 --2- 172.21.15.143:0/1506057330 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f975c09d740 0x7f975c14ffb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.234+0000 7f975b59e700 1 -- 172.21.15.143:0/1506057330 >> 172.21.15.143:0/1506057330 conn(0x7f975c0a59b0 msgr2=0x7f975c0a1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.238+0000 7f975b59e700 1 -- 172.21.15.143:0/1506057330 shutdown_connections 2023-09-29T23:30:51.241 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:51.238+0000 7f975b59e700 1 -- 172.21.15.143:0/1506057330 wait complete. 2023-09-29T23:30:54.254 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:54.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f4484e26700 1 Processor -- start 2023-09-29T23:30:54.367 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f4484e26700 1 -- start start 2023-09-29T23:30:54.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f4484e26700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f44800a1170 0x7f448013d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:54.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f4484e26700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448009ff40 con 0x7f4480066ec0 2023-09-29T23:30:54.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f4484e26700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4480067290 con 0x7f448013d680 2023-09-29T23:30:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f4484e26700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44800a0510 con 0x7f44800a1170 2023-09-29T23:30:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f447f7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4480066ec0 0x7f448009f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33264/0 (socket says 172.21.15.143:33264) 2023-09-29T23:30:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f447f7fe700 1 -- 172.21.15.143:0/4246624138 learned_addr learned my addr 172.21.15.143:0/4246624138 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:30:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f447effd700 1 --2- 172.21.15.143:0/4246624138 >> v2:172.21.15.156:3300/0 conn(0x7f44800a1170 0x7f448013d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f447e7fc700 1 -- 172.21.15.143:0/4246624138 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580001995 0 0) 0x7f448009ff40 con 0x7f4480066ec0 2023-09-29T23:30:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f447e7fc700 1 -- 172.21.15.143:0/4246624138 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4468003570 con 0x7f4480066ec0 2023-09-29T23:30:54.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f447effd700 1 -- 172.21.15.143:0/4246624138 >> v1:172.21.15.156:6789/0 conn(0x7f448013d680 legacy=0x7f4480148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:54.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f447effd700 1 -- 172.21.15.143:0/4246624138 >> v1:172.21.15.143:6789/0 conn(0x7f4480066ec0 legacy=0x7f448009f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:54.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.362+0000 7f447effd700 1 -- 172.21.15.143:0/4246624138 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448006e400 con 0x7f44800a1170 2023-09-29T23:30:54.370 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.366+0000 7f447effd700 1 --2- 172.21.15.143:0/4246624138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44800a1170 0x7f448013d140 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4470009e60 tx=0x7f4470009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcf35b912636bb91 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.366+0000 7f447e7fc700 1 -- 172.21.15.143:0/4246624138 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447000e070 con 0x7f44800a1170 2023-09-29T23:30:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.366+0000 7f447e7fc700 1 -- 172.21.15.143:0/4246624138 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447000bb60 con 0x7f44800a1170 2023-09-29T23:30:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.366+0000 7f447e7fc700 1 -- 172.21.15.143:0/4246624138 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470012430 con 0x7f44800a1170 2023-09-29T23:30:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.366+0000 7f4484e26700 1 -- 172.21.15.143:0/4246624138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44800a1170 msgr2=0x7f448013d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.366+0000 7f4484e26700 1 --2- 172.21.15.143:0/4246624138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44800a1170 0x7f448013d140 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4470009e60 tx=0x7f4470009780 comp rx=0 tx=0).stop 2023-09-29T23:30:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.366+0000 7f4484e26700 1 -- 172.21.15.143:0/4246624138 shutdown_connections 2023-09-29T23:30:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.366+0000 7f4484e26700 1 --2- 172.21.15.143:0/4246624138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44800a1170 0x7f448013d140 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.366+0000 7f4484e26700 1 -- 172.21.15.143:0/4246624138 >> 172.21.15.143:0/4246624138 conn(0x7f448006f200 msgr2=0x7f4480069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:54.371 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.366+0000 7f4484e26700 1 -- 172.21.15.143:0/4246624138 shutdown_connections 2023-09-29T23:30:54.372 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.366+0000 7f4484e26700 1 -- 172.21.15.143:0/4246624138 wait complete. 2023-09-29T23:30:54.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f4484e26700 1 Processor -- start 2023-09-29T23:30:54.373 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f4484e26700 1 -- start start 2023-09-29T23:30:54.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f4484e26700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4480066ec0 0x7f44800990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:54.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f4484e26700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44800a1170 0x7f4480140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:54.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f4484e26700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f448013d680 0x7f4480140c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:54.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f4484e26700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44801411a0 con 0x7f44800a1170 2023-09-29T23:30:54.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f4484e26700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44801402d0 con 0x7f448013d680 2023-09-29T23:30:54.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f4484e26700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4480095ab0 con 0x7f4480066ec0 2023-09-29T23:30:54.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f447f7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4480066ec0 0x7f44800990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:54.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f447effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44800a1170 0x7f4480140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:54.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f447f7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4480066ec0 0x7f44800990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:35258/0 (socket says 172.21.15.143:35258) 2023-09-29T23:30:54.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f447f7fe700 1 -- 172.21.15.143:0/4176877372 learned_addr learned my addr 172.21.15.143:0/4176877372 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:54.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f447ffff700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f448013d680 0x7f4480140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:54.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f447effd700 1 -- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4480066ec0 msgr2=0x7f44800990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:54.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f447effd700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4480066ec0 0x7f44800990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:54.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f447effd700 1 -- 172.21.15.143:0/4176877372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f448013d680 msgr2=0x7f4480140c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:54.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f447effd700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f448013d680 0x7f4480140c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:54.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f447effd700 1 -- 172.21.15.143:0/4176877372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4470009a20 con 0x7f44800a1170 2023-09-29T23:30:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f447effd700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44800a1170 0x7f4480140720 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4470016000 tx=0x7f4470019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f44677fe700 1 -- 172.21.15.143:0/4176877372 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470012970 con 0x7f44800a1170 2023-09-29T23:30:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f4484e26700 1 -- 172.21.15.143:0/4176877372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44801413b0 con 0x7f44800a1170 2023-09-29T23:30:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f44677fe700 1 -- 172.21.15.143:0/4176877372 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4470021070 con 0x7f44800a1170 2023-09-29T23:30:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f44677fe700 1 -- 172.21.15.143:0/4176877372 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470012d60 con 0x7f44800a1170 2023-09-29T23:30:54.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.370+0000 7f4484e26700 1 -- 172.21.15.143:0/4176877372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44801397a0 con 0x7f44800a1170 2023-09-29T23:30:54.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.374+0000 7f4484e26700 1 -- 172.21.15.143:0/4176877372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448009f550 con 0x7f44800a1170 2023-09-29T23:30:54.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.374+0000 7f44677fe700 1 -- 172.21.15.143:0/4176877372 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f4470023030 con 0x7f44800a1170 2023-09-29T23:30:54.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.374+0000 7f44677fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:54.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.374+0000 7f44677fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:54.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.374+0000 7f44677fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:54.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.374+0000 7f44677fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:54.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.374+0000 7f44677fe700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f446c08e830 0x7f446c090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:54.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.378+0000 7f447f7fe700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f446c08e830 0x7f446c090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:54.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.378+0000 7f44677fe700 1 -- 172.21.15.143:0/4176877372 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f44700b6a10 con 0x7f44800a1170 2023-09-29T23:30:54.381 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.378+0000 7f447f7fe700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f446c08e830 0x7f446c090ce0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4474001ad0 tx=0x7f4474009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:54.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.378+0000 7f44677fe700 1 -- 172.21.15.143:0/4176877372 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4470087460 con 0x7f44800a1170 2023-09-29T23:30:54.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 -- 172.21.15.143:0/4176877372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f448009f550 con 0x7f44800a1170 2023-09-29T23:30:54.578 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f44677fe700 1 -- 172.21.15.143:0/4176877372 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f448009f550 con 0x7f44800a1170 2023-09-29T23:30:54.578 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:54.578 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:54.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 10 mgrc shutdown 2023-09-29T23:30:54.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 -- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f446c08e830 msgr2=0x7f446c090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:54.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f446c08e830 0x7f446c090ce0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4474001ad0 tx=0x7f4474009040 comp rx=0 tx=0).stop 2023-09-29T23:30:54.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 -- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44800a1170 msgr2=0x7f4480140720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:54.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44800a1170 0x7f4480140720 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4470016000 tx=0x7f4470019000 comp rx=0 tx=0).stop 2023-09-29T23:30:54.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 -- 172.21.15.143:0/4176877372 shutdown_connections 2023-09-29T23:30:54.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f446c08e830 0x7f446c090ce0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:54.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f448013d680 0x7f4480140c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:54.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44800a1170 0x7f4480140720 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:54.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 --2- 172.21.15.143:0/4176877372 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4480066ec0 0x7f44800990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:54.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 -- 172.21.15.143:0/4176877372 >> 172.21.15.143:0/4176877372 conn(0x7f448006f200 msgr2=0x7f448013c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:54.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 -- 172.21.15.143:0/4176877372 shutdown_connections 2023-09-29T23:30:54.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:54.574+0000 7f4484e26700 1 -- 172.21.15.143:0/4176877372 wait complete. 2023-09-29T23:30:55.970 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:30:55.972 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:30:57.593 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:30:57.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21c4c93700 1 Processor -- start 2023-09-29T23:30:57.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21c4c93700 1 -- start start 2023-09-29T23:30:57.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21c4c93700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f21c006ba00 0x7f21c009f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:57.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21c4c93700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f21c006be40 0x7f21c0146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:57.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21c4c93700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21c006c240 con 0x7f21c0132640 2023-09-29T23:30:57.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21c4c93700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c00a0460 con 0x7f21c006ba00 2023-09-29T23:30:57.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21c4c93700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21c009fee0 con 0x7f21c006be40 2023-09-29T23:30:57.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21bf7fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f21c0132640 0x7f21c0067520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33268/0 (socket says 172.21.15.143:33268) 2023-09-29T23:30:57.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21bf7fe700 1 -- 172.21.15.143:0/2339169317 learned_addr learned my addr 172.21.15.143:0/2339169317 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:30:57.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21beffd700 1 --2- 172.21.15.143:0/2339169317 >> v2:172.21.15.143:3300/0 conn(0x7f21c006ba00 0x7f21c009f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:57.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21bffff700 1 --2- 172.21.15.143:0/2339169317 >> v2:172.21.15.156:3300/0 conn(0x7f21c006be40 0x7f21c0146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:57.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21be7fc700 1 -- 172.21.15.143:0/2339169317 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 85347732 0 0) 0x7f21c006c240 con 0x7f21c0132640 2023-09-29T23:30:57.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21be7fc700 1 -- 172.21.15.143:0/2339169317 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21a8003570 con 0x7f21c0132640 2023-09-29T23:30:57.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21beffd700 1 -- 172.21.15.143:0/2339169317 >> v2:172.21.15.156:3300/0 conn(0x7f21c006be40 msgr2=0x7f21c0146bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:30:57.710 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21beffd700 1 --2- 172.21.15.143:0/2339169317 >> v2:172.21.15.156:3300/0 conn(0x7f21c006be40 0x7f21c0146bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21beffd700 1 -- 172.21.15.143:0/2339169317 >> v1:172.21.15.143:6789/0 conn(0x7f21c0132640 legacy=0x7f21c0067520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:57.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21be7fc700 1 -- 172.21.15.143:0/2339169317 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1017235824 0 0) 0x7f21a8003570 con 0x7f21c0132640 2023-09-29T23:30:57.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.702+0000 7f21beffd700 1 -- 172.21.15.143:0/2339169317 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c00a01a0 con 0x7f21c006ba00 2023-09-29T23:30:57.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21beffd700 1 --2- 172.21.15.143:0/2339169317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c006ba00 0x7f21c009f550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f21c0068580 tx=0x7f21b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d71e69dac0c8bc server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:57.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21be7fc700 1 -- 172.21.15.143:0/2339169317 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b80017f0 con 0x7f21c006ba00 2023-09-29T23:30:57.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21be7fc700 1 -- 172.21.15.143:0/2339169317 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b400f4a0 con 0x7f21c006ba00 2023-09-29T23:30:57.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21be7fc700 1 -- 172.21.15.143:0/2339169317 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b400f840 con 0x7f21c006ba00 2023-09-29T23:30:57.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 -- 172.21.15.143:0/2339169317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c006ba00 msgr2=0x7f21c009f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:57.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 --2- 172.21.15.143:0/2339169317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c006ba00 0x7f21c009f550 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f21c0068580 tx=0x7f21b40091a0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 -- 172.21.15.143:0/2339169317 shutdown_connections 2023-09-29T23:30:57.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 --2- 172.21.15.143:0/2339169317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c006ba00 0x7f21c009f550 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 --2- 172.21.15.143:0/2339169317 >> v2:172.21.15.156:3300/0 conn(0x7f21c006be40 0x7f21c0146bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 -- 172.21.15.143:0/2339169317 >> 172.21.15.143:0/2339169317 conn(0x7f21c006f200 msgr2=0x7f21c0069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:57.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 -- 172.21.15.143:0/2339169317 shutdown_connections 2023-09-29T23:30:57.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 -- 172.21.15.143:0/2339169317 wait complete. 2023-09-29T23:30:57.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 Processor -- start 2023-09-29T23:30:57.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 -- start start 2023-09-29T23:30:57.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c006ba00 0x7f21c0143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:57.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0132640 0x7f21c012dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:57.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f21c012f460 0x7f21c0153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:57.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c012e9f0 con 0x7f21c0132640 2023-09-29T23:30:57.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c012d280 con 0x7f21c006ba00 2023-09-29T23:30:57.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21c4c93700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c012d540 con 0x7f21c012f460 2023-09-29T23:30:57.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21bffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f21c012f460 0x7f21c0153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:57.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21bffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f21c012f460 0x7f21c0153320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:35260/0 (socket says 172.21.15.143:35260) 2023-09-29T23:30:57.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21bffff700 1 -- 172.21.15.143:0/4046863354 learned_addr learned my addr 172.21.15.143:0/4046863354 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:30:57.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21beffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0132640 0x7f21c012dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:57.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21bf7fe700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c006ba00 0x7f21c0143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:57.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21bffff700 1 -- 172.21.15.143:0/4046863354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c006ba00 msgr2=0x7f21c0143380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:57.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21bffff700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c006ba00 0x7f21c0143380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21bffff700 1 -- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0132640 msgr2=0x7f21c012dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:57.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21bffff700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0132640 0x7f21c012dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21bffff700 1 -- 172.21.15.143:0/4046863354 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b4009420 con 0x7f21c012f460 2023-09-29T23:30:57.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21bffff700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f21c012f460 0x7f21c0153320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f21b000ea50 tx=0x7f21b00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:57.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.706+0000 7f21a77fe700 1 -- 172.21.15.143:0/4046863354 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b000c850 con 0x7f21c012f460 2023-09-29T23:30:57.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.710+0000 7f21a77fe700 1 -- 172.21.15.143:0/4046863354 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21b000c9d0 con 0x7f21c012f460 2023-09-29T23:30:57.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.710+0000 7f21c4c93700 1 -- 172.21.15.143:0/4046863354 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c0140690 con 0x7f21c012f460 2023-09-29T23:30:57.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.710+0000 7f21a77fe700 1 -- 172.21.15.143:0/4046863354 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21b000cd90 con 0x7f21c012f460 2023-09-29T23:30:57.719 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.710+0000 7f21c4c93700 1 -- 172.21.15.143:0/4046863354 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c012e290 con 0x7f21c012f460 2023-09-29T23:30:57.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.710+0000 7f21c4c93700 1 -- 172.21.15.143:0/4046863354 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c012f180 con 0x7f21c012f460 2023-09-29T23:30:57.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.714+0000 7f21a77fe700 1 -- 172.21.15.143:0/4046863354 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f21b0013680 con 0x7f21c012f460 2023-09-29T23:30:57.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.714+0000 7f21a77fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:30:57.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.714+0000 7f21a77fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:30:57.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.714+0000 7f21a77fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:57.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.714+0000 7f21a77fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:30:57.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.714+0000 7f21a77fe700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f21ac08e8b0 0x7f21ac090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:30:57.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.714+0000 7f21bf7fe700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f21ac08e8b0 0x7f21ac090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:30:57.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.714+0000 7f21a77fe700 1 -- 172.21.15.143:0/4046863354 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f21b00b5190 con 0x7f21c012f460 2023-09-29T23:30:57.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.714+0000 7f21bf7fe700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f21ac08e8b0 0x7f21ac090d60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f21b8001560 tx=0x7f21b8007ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:30:57.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.718+0000 7f21a77fe700 1 -- 172.21.15.143:0/4046863354 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f21b0085ab0 con 0x7f21c012f460 2023-09-29T23:30:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.914+0000 7f21c4c93700 1 -- 172.21.15.143:0/4046863354 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f21c012f160 con 0x7f21c012f460 2023-09-29T23:30:57.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.914+0000 7f21a77fe700 1 -- 172.21.15.143:0/4046863354 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f21b0017100 con 0x7f21c012f460 2023-09-29T23:30:57.920 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:30:57.920 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:30:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 10 mgrc shutdown 2023-09-29T23:30:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 -- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f21ac08e8b0 msgr2=0x7f21ac090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f21ac08e8b0 0x7f21ac090d60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f21b8001560 tx=0x7f21b8007ef0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 -- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f21c012f460 msgr2=0x7f21c0153320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:30:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f21c012f460 0x7f21c0153320 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f21b000ea50 tx=0x7f21b00091a0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 -- 172.21.15.143:0/4046863354 shutdown_connections 2023-09-29T23:30:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f21ac08e8b0 0x7f21ac090d60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f21c012f460 0x7f21c0153320 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f21c0132640 0x7f21c012dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 --2- 172.21.15.143:0/4046863354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21c006ba00 0x7f21c0143380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:30:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 -- 172.21.15.143:0/4046863354 >> 172.21.15.143:0/4046863354 conn(0x7f21c006f200 msgr2=0x7f21c00a30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:30:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 -- 172.21.15.143:0/4046863354 shutdown_connections 2023-09-29T23:30:57.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:30:57.918+0000 7f21c4c93700 1 -- 172.21.15.143:0/4046863354 wait complete. 2023-09-29T23:31:00.939 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.046+0000 7f777bfff700 1 Processor -- start 2023-09-29T23:31:01.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.046+0000 7f777bfff700 1 -- start start 2023-09-29T23:31:01.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f777c09ebd0 0x7f777c09d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f777c09dcb0 0x7f777c13c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f777c0b3ae0 con 0x7f777c0b3710 2023-09-29T23:31:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777c0a0090 con 0x7f777c09dcb0 2023-09-29T23:31:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777c09f0e0 con 0x7f777c09ebd0 2023-09-29T23:31:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777affd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f777c0b3710 0x7f777c0a58d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33284/0 (socket says 172.21.15.143:33284) 2023-09-29T23:31:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777affd700 1 -- 172.21.15.143:0/1944059826 learned_addr learned my addr 172.21.15.143:0/1944059826 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:31:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777b7fe700 1 --2- 172.21.15.143:0/1944059826 >> v2:172.21.15.143:3300/0 conn(0x7f777c09dcb0 0x7f777c13c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777a7fc700 1 --2- 172.21.15.143:0/1944059826 >> v2:172.21.15.156:3300/0 conn(0x7f777c09ebd0 0x7f777c09d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f7779ffb700 1 -- 172.21.15.143:0/1944059826 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091214239 0 0) 0x7f777c0b3ae0 con 0x7f777c0b3710 2023-09-29T23:31:01.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f7779ffb700 1 -- 172.21.15.143:0/1944059826 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7768003570 con 0x7f777c0b3710 2023-09-29T23:31:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f7779ffb700 1 -- 172.21.15.143:0/1944059826 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1951043119 0 0) 0x7f7768003570 con 0x7f777c0b3710 2023-09-29T23:31:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777b7fe700 1 -- 172.21.15.143:0/1944059826 >> v2:172.21.15.156:3300/0 conn(0x7f777c09ebd0 msgr2=0x7f777c09d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777b7fe700 1 --2- 172.21.15.143:0/1944059826 >> v2:172.21.15.156:3300/0 conn(0x7f777c09ebd0 0x7f777c09d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777b7fe700 1 -- 172.21.15.143:0/1944059826 >> v1:172.21.15.143:6789/0 conn(0x7f777c0b3710 legacy=0x7f777c0a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:01.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777b7fe700 1 -- 172.21.15.143:0/1944059826 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f777c09f3a0 con 0x7f777c09dcb0 2023-09-29T23:31:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777b7fe700 1 --2- 172.21.15.143:0/1944059826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777c09dcb0 0x7f777c13c460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f777c0b4420 tx=0x7f776c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78578f04178026d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f7779ffb700 1 -- 172.21.15.143:0/1944059826 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77740017f0 con 0x7f777c09dcb0 2023-09-29T23:31:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f7779ffb700 1 -- 172.21.15.143:0/1944059826 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c011420 con 0x7f777c09dcb0 2023-09-29T23:31:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f7779ffb700 1 -- 172.21.15.143:0/1944059826 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c0117c0 con 0x7f777c09dcb0 2023-09-29T23:31:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 -- 172.21.15.143:0/1944059826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777c09dcb0 msgr2=0x7f777c13c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:01.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 --2- 172.21.15.143:0/1944059826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777c09dcb0 0x7f777c13c460 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f777c0b4420 tx=0x7f776c0091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:01.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 -- 172.21.15.143:0/1944059826 shutdown_connections 2023-09-29T23:31:01.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 --2- 172.21.15.143:0/1944059826 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777c09dcb0 0x7f777c13c460 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:01.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 --2- 172.21.15.143:0/1944059826 >> v2:172.21.15.156:3300/0 conn(0x7f777c09ebd0 0x7f777c09d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:01.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 -- 172.21.15.143:0/1944059826 >> 172.21.15.143:0/1944059826 conn(0x7f777c062ed0 msgr2=0x7f777c0632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:01.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 -- 172.21.15.143:0/1944059826 shutdown_connections 2023-09-29T23:31:01.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.050+0000 7f777bfff700 1 -- 172.21.15.143:0/1944059826 wait complete. 2023-09-29T23:31:01.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777bfff700 1 Processor -- start 2023-09-29T23:31:01.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777bfff700 1 -- start start 2023-09-29T23:31:01.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f777c09dcb0 0x7f777c06be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:01.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f777c09ebd0 0x7f777c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:01.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777c0b3710 0x7f777c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:01.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777bfff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c06db40 con 0x7f777c0b3710 2023-09-29T23:31:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777bfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777c0a2fa0 con 0x7f777c09dcb0 2023-09-29T23:31:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777bfff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777c14c930 con 0x7f777c09ebd0 2023-09-29T23:31:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777a7fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f777c09ebd0 0x7f777c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777c0b3710 0x7f777c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777a7fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f777c09ebd0 0x7f777c06d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:35274/0 (socket says 172.21.15.143:35274) 2023-09-29T23:31:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777a7fc700 1 -- 172.21.15.143:0/1936843385 learned_addr learned my addr 172.21.15.143:0/1936843385 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:01.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.054+0000 7f777affd700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f777c09dcb0 0x7f777c06be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f777b7fe700 1 -- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f777c09ebd0 msgr2=0x7f777c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f777b7fe700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f777c09ebd0 0x7f777c06d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:01.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f777b7fe700 1 -- 172.21.15.143:0/1936843385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f777c09dcb0 msgr2=0x7f777c06be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f777b7fe700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f777c09dcb0 0x7f777c06be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f777b7fe700 1 -- 172.21.15.143:0/1936843385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f776c009420 con 0x7f777c0b3710 2023-09-29T23:31:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f777b7fe700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777c0b3710 0x7f777c06d600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f776c015000 tx=0x7f776c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f7772ffd700 1 -- 172.21.15.143:0/1936843385 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c01cc00 con 0x7f777c0b3710 2023-09-29T23:31:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f7772ffd700 1 -- 172.21.15.143:0/1936843385 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c01cd80 con 0x7f777c0b3710 2023-09-29T23:31:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f777bfff700 1 -- 172.21.15.143:0/1936843385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f777c13bcb0 con 0x7f777c0b3710 2023-09-29T23:31:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f7772ffd700 1 -- 172.21.15.143:0/1936843385 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c019640 con 0x7f777c0b3710 2023-09-29T23:31:01.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f777bfff700 1 -- 172.21.15.143:0/1936843385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f777c06c360 con 0x7f777c0b3710 2023-09-29T23:31:01.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f7772ffd700 1 -- 172.21.15.143:0/1936843385 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f776c019a70 con 0x7f777c0b3710 2023-09-29T23:31:01.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f7772ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:01.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.058+0000 7f777bfff700 1 -- 172.21.15.143:0/1936843385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777c05ecb0 con 0x7f777c0b3710 2023-09-29T23:31:01.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.062+0000 7f7772ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:01.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.062+0000 7f7772ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:01.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.062+0000 7f7772ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:01.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.062+0000 7f7772ffd700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f775c08e8b0 0x7f775c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:01.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.062+0000 7f7772ffd700 1 -- 172.21.15.143:0/1936843385 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f776c0b5ac0 con 0x7f777c0b3710 2023-09-29T23:31:01.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.062+0000 7f777affd700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f775c08e8b0 0x7f775c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:01.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.062+0000 7f777affd700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f775c08e8b0 0x7f775c090d60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7774001560 tx=0x7f7774001640 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:01.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.066+0000 7f7772ffd700 1 -- 172.21.15.143:0/1936843385 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f776c086510 con 0x7f777c0b3710 2023-09-29T23:31:01.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.262+0000 7f777bfff700 1 -- 172.21.15.143:0/1936843385 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f777c0a0d60 con 0x7f777c0b3710 2023-09-29T23:31:01.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.262+0000 7f7772ffd700 1 -- 172.21.15.143:0/1936843385 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f776c089b30 con 0x7f777c0b3710 2023-09-29T23:31:01.267 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:01.268 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:01.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 10 mgrc shutdown 2023-09-29T23:31:01.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 -- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f775c08e8b0 msgr2=0x7f775c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:01.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f775c08e8b0 0x7f775c090d60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7774001560 tx=0x7f7774001640 comp rx=0 tx=0).stop 2023-09-29T23:31:01.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 -- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777c0b3710 msgr2=0x7f777c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:01.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777c0b3710 0x7f777c06d600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f776c015000 tx=0x7f776c016000 comp rx=0 tx=0).stop 2023-09-29T23:31:01.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 -- 172.21.15.143:0/1936843385 shutdown_connections 2023-09-29T23:31:01.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f775c08e8b0 0x7f775c090d60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:01.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f777c0b3710 0x7f777c06d600 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:01.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f777c09ebd0 0x7f777c06d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:01.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 --2- 172.21.15.143:0/1936843385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f777c09dcb0 0x7f777c06be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:01.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 -- 172.21.15.143:0/1936843385 >> 172.21.15.143:0/1936843385 conn(0x7f777c062ed0 msgr2=0x7f777c06fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:01.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 -- 172.21.15.143:0/1936843385 shutdown_connections 2023-09-29T23:31:01.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:01.266+0000 7f777bfff700 1 -- 172.21.15.143:0/1936843385 wait complete. 2023-09-29T23:31:04.282 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:04.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.390+0000 7f6259533700 1 Processor -- start 2023-09-29T23:31:04.395 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.390+0000 7f6259533700 1 -- start start 2023-09-29T23:31:04.396 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.390+0000 7f6259533700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f62540a1170 0x7f625413d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:04.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.390+0000 7f6259533700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f625413d680 0x7f6254148590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:04.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.390+0000 7f6259533700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f625409ff40 con 0x7f6254066ec0 2023-09-29T23:31:04.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.390+0000 7f6259533700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6254067290 con 0x7f625413d680 2023-09-29T23:31:04.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.390+0000 7f6259533700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62540a0510 con 0x7f62540a1170 2023-09-29T23:31:04.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.390+0000 7f62537fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f62540a1170 0x7f625413d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:04.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.390+0000 7f62537fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f62540a1170 0x7f625413d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:45286/0 (socket says 172.21.15.143:45286) 2023-09-29T23:31:04.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.390+0000 7f62537fe700 1 -- 172.21.15.143:0/3730680215 learned_addr learned my addr 172.21.15.143:0/3730680215 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:04.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.390+0000 7f6258d32700 1 --2- 172.21.15.143:0/3730680215 >> v2:172.21.15.143:3300/0 conn(0x7f625413d680 0x7f6254148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:04.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f62537fe700 1 -- 172.21.15.143:0/3730680215 >> v2:172.21.15.143:3300/0 conn(0x7f625413d680 msgr2=0x7f6254148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:04.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f62537fe700 1 --2- 172.21.15.143:0/3730680215 >> v2:172.21.15.143:3300/0 conn(0x7f625413d680 0x7f6254148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:04.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f62537fe700 1 -- 172.21.15.143:0/3730680215 >> v1:172.21.15.156:6789/0 conn(0x7f6254066ec0 legacy=0x7f625409f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:04.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f62537fe700 1 -- 172.21.15.143:0/3730680215 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625406e400 con 0x7f62540a1170 2023-09-29T23:31:04.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f62537fe700 1 --2- 172.21.15.143:0/3730680215 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f62540a1170 0x7f625413d140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6244009f20 tx=0x7f6244009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94dab32f70eb7e76 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:04.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6252ffd700 1 -- 172.21.15.143:0/3730680215 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624400e070 con 0x7f62540a1170 2023-09-29T23:31:04.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6252ffd700 1 -- 172.21.15.143:0/3730680215 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624400b9d0 con 0x7f62540a1170 2023-09-29T23:31:04.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6252ffd700 1 -- 172.21.15.143:0/3730680215 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62440092e0 con 0x7f62540a1170 2023-09-29T23:31:04.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 -- 172.21.15.143:0/3730680215 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f62540a1170 msgr2=0x7f625413d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:04.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 --2- 172.21.15.143:0/3730680215 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f62540a1170 0x7f625413d140 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6244009f20 tx=0x7f6244009840 comp rx=0 tx=0).stop 2023-09-29T23:31:04.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 -- 172.21.15.143:0/3730680215 shutdown_connections 2023-09-29T23:31:04.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 --2- 172.21.15.143:0/3730680215 >> v2:172.21.15.143:3300/0 conn(0x7f625413d680 0x7f6254148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:04.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 --2- 172.21.15.143:0/3730680215 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f62540a1170 0x7f625413d140 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:04.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 -- 172.21.15.143:0/3730680215 >> 172.21.15.143:0/3730680215 conn(0x7f625406f200 msgr2=0x7f6254069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:04.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 -- 172.21.15.143:0/3730680215 shutdown_connections 2023-09-29T23:31:04.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 -- 172.21.15.143:0/3730680215 wait complete. 2023-09-29T23:31:04.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 Processor -- start 2023-09-29T23:31:04.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 -- start start 2023-09-29T23:31:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6254066ec0 0x7f625406c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62540a1170 0x7f62541416a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f625413d680 0x7f625409c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625409bca0 con 0x7f62540a1170 2023-09-29T23:31:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625409ca60 con 0x7f6254066ec0 2023-09-29T23:31:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6259533700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625409c6e0 con 0x7f625413d680 2023-09-29T23:31:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6258d32700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f625413d680 0x7f625409c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f62537fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62540a1170 0x7f62541416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6258d32700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f625413d680 0x7f625409c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:45298/0 (socket says 172.21.15.143:45298) 2023-09-29T23:31:04.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f62537fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62540a1170 0x7f62541416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43370/0 (socket says 172.21.15.143:43370) 2023-09-29T23:31:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6258d32700 1 -- 172.21.15.143:0/656394729 learned_addr learned my addr 172.21.15.143:0/656394729 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.394+0000 7f6253fff700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6254066ec0 0x7f625406c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f6258d32700 1 -- 172.21.15.143:0/656394729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6254066ec0 msgr2=0x7f625406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f6258d32700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6254066ec0 0x7f625406c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f6258d32700 1 -- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62540a1170 msgr2=0x7f62541416a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f6258d32700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62540a1170 0x7f62541416a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f6258d32700 1 -- 172.21.15.143:0/656394729 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6244009ae0 con 0x7f625413d680 2023-09-29T23:31:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f62537fe700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62540a1170 0x7f62541416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f6258d32700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f625413d680 0x7f625409c1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f624c00e8d0 tx=0x7f624c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f623bfff700 1 -- 172.21.15.143:0/656394729 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c013680 con 0x7f625413d680 2023-09-29T23:31:04.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f623bfff700 1 -- 172.21.15.143:0/656394729 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c0137e0 con 0x7f625413d680 2023-09-29T23:31:04.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f6259533700 1 -- 172.21.15.143:0/656394729 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6254146040 con 0x7f625413d680 2023-09-29T23:31:04.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f623bfff700 1 -- 172.21.15.143:0/656394729 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c013ba0 con 0x7f625413d680 2023-09-29T23:31:04.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f6259533700 1 -- 172.21.15.143:0/656394729 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62541462e0 con 0x7f625413d680 2023-09-29T23:31:04.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.398+0000 7f6259533700 1 -- 172.21.15.143:0/656394729 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625409b560 con 0x7f625413d680 2023-09-29T23:31:04.404 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.402+0000 7f623bfff700 1 -- 172.21.15.143:0/656394729 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f624c0253d0 con 0x7f625413d680 2023-09-29T23:31:04.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.402+0000 7f623bfff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:04.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.402+0000 7f623bfff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:04.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.402+0000 7f623bfff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:04.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.402+0000 7f623bfff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:04.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.402+0000 7f623bfff700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f624008e8b0 0x7f6240090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:04.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.402+0000 7f6253fff700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f624008e8b0 0x7f6240090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:04.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.402+0000 7f623bfff700 1 -- 172.21.15.143:0/656394729 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f624c0b4e70 con 0x7f625413d680 2023-09-29T23:31:04.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.402+0000 7f6253fff700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f624008e8b0 0x7f6240090d60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6248003a80 tx=0x7f6248007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:04.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.406+0000 7f623bfff700 1 -- 172.21.15.143:0/656394729 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f624c0858c0 con 0x7f625413d680 2023-09-29T23:31:04.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.602+0000 7f6259533700 1 -- 172.21.15.143:0/656394729 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f625409b560 con 0x7f625413d680 2023-09-29T23:31:04.607 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.602+0000 7f623bfff700 1 -- 172.21.15.143:0/656394729 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f624c017100 con 0x7f625413d680 2023-09-29T23:31:04.607 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:04.607 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:04.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 10 mgrc shutdown 2023-09-29T23:31:04.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 -- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f624008e8b0 msgr2=0x7f6240090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:04.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f624008e8b0 0x7f6240090d60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6248003a80 tx=0x7f6248007ec0 comp rx=0 tx=0).stop 2023-09-29T23:31:04.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 -- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f625413d680 msgr2=0x7f625409c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:04.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f625413d680 0x7f625409c1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f624c00e8d0 tx=0x7f624c0091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:04.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 -- 172.21.15.143:0/656394729 shutdown_connections 2023-09-29T23:31:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f624008e8b0 0x7f6240090d60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f625413d680 0x7f625409c1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62540a1170 0x7f62541416a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 --2- 172.21.15.143:0/656394729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6254066ec0 0x7f625406c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 -- 172.21.15.143:0/656394729 >> 172.21.15.143:0/656394729 conn(0x7f625406f200 msgr2=0x7f625413a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 -- 172.21.15.143:0/656394729 shutdown_connections 2023-09-29T23:31:04.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:04.606+0000 7f6259533700 1 -- 172.21.15.143:0/656394729 wait complete. 2023-09-29T23:31:07.628 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:07.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19f4ac7700 1 Processor -- start 2023-09-29T23:31:07.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19f4ac7700 1 -- start start 2023-09-29T23:31:07.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19f4ac7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f19f0063100 0x7f19f0135ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19f4ac7700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f19f00b3e90 0x7f19f01363e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19f4ac7700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f009ec70 con 0x7f19f009fdd0 2023-09-29T23:31:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19f4ac7700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f00b4410 con 0x7f19f0063100 2023-09-29T23:31:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19f4ac7700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f009f1a0 con 0x7f19f00b3e90 2023-09-29T23:31:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19ef7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f19f0063100 0x7f19f0135ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19e7fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f19f009fdd0 0x7f19f00b3780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36136/0 (socket says 172.21.15.143:36136) 2023-09-29T23:31:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19e7fff700 1 -- 172.21.15.143:0/1338179348 learned_addr learned my addr 172.21.15.143:0/1338179348 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:31:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19effff700 1 --2- 172.21.15.143:0/1338179348 >> v2:172.21.15.156:3300/0 conn(0x7f19f00b3e90 0x7f19f01363e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19ef7fe700 1 -- 172.21.15.143:0/1338179348 >> v2:172.21.15.156:3300/0 conn(0x7f19f00b3e90 msgr2=0x7f19f01363e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19ef7fe700 1 --2- 172.21.15.143:0/1338179348 >> v2:172.21.15.156:3300/0 conn(0x7f19f00b3e90 0x7f19f01363e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:07.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19ef7fe700 1 -- 172.21.15.143:0/1338179348 >> v1:172.21.15.143:6789/0 conn(0x7f19f009fdd0 legacy=0x7f19f00b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:07.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19ef7fe700 1 -- 172.21.15.143:0/1338179348 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f009f3a0 con 0x7f19f0063100 2023-09-29T23:31:07.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19effff700 1 --2- 172.21.15.143:0/1338179348 >> v2:172.21.15.156:3300/0 conn(0x7f19f00b3e90 0x7f19f01363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:31:07.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19ef7fe700 1 --2- 172.21.15.143:0/1338179348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19f0063100 0x7f19f0135ea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f19f00a2f90 tx=0x7f19d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5909a1d32a996391 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:07.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19eeffd700 1 -- 172.21.15.143:0/1338179348 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d800b6b0 con 0x7f19f0063100 2023-09-29T23:31:07.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19eeffd700 1 -- 172.21.15.143:0/1338179348 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d800b810 con 0x7f19f0063100 2023-09-29T23:31:07.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.738+0000 7f19eeffd700 1 -- 172.21.15.143:0/1338179348 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d800bbb0 con 0x7f19f0063100 2023-09-29T23:31:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1338179348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19f0063100 msgr2=0x7f19f0135ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 --2- 172.21.15.143:0/1338179348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19f0063100 0x7f19f0135ea0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f19f00a2f90 tx=0x7f19d80091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1338179348 shutdown_connections 2023-09-29T23:31:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 --2- 172.21.15.143:0/1338179348 >> v2:172.21.15.156:3300/0 conn(0x7f19f00b3e90 0x7f19f01363e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 --2- 172.21.15.143:0/1338179348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19f0063100 0x7f19f0135ea0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:07.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1338179348 >> 172.21.15.143:0/1338179348 conn(0x7f19f006ab30 msgr2=0x7f19f006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:07.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1338179348 shutdown_connections 2023-09-29T23:31:07.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1338179348 wait complete. 2023-09-29T23:31:07.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 Processor -- start 2023-09-29T23:31:07.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 -- start start 2023-09-29T23:31:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f0063100 0x7f19f012c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19f009fdd0 0x7f19f012a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f19f00b3e90 0x7f19f012a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f012ad40 con 0x7f19f009fdd0 2023-09-29T23:31:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f012af30 con 0x7f19f0063100 2023-09-29T23:31:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19f4ac7700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f012b1d0 con 0x7f19f00b3e90 2023-09-29T23:31:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19effff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f19f00b3e90 0x7f19f012a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19e7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19f009fdd0 0x7f19f012a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19effff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f19f00b3e90 0x7f19f012a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:45314/0 (socket says 172.21.15.143:45314) 2023-09-29T23:31:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19effff700 1 -- 172.21.15.143:0/1764812773 learned_addr learned my addr 172.21.15.143:0/1764812773 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:07.749 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19ef7fe700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f0063100 0x7f19f012c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19effff700 1 -- 172.21.15.143:0/1764812773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f0063100 msgr2=0x7f19f012c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19effff700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f0063100 0x7f19f012c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19effff700 1 -- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19f009fdd0 msgr2=0x7f19f012a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19effff700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19f009fdd0 0x7f19f012a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19effff700 1 -- 172.21.15.143:0/1764812773 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d8009420 con 0x7f19f00b3e90 2023-09-29T23:31:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.742+0000 7f19e7fff700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19f009fdd0 0x7f19f012a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.746+0000 7f19effff700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f19f00b3e90 0x7f19f012a800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009fc0 tx=0x7f19e80099b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.746+0000 7f19e77fe700 1 -- 172.21.15.143:0/1764812773 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e800dbe0 con 0x7f19f00b3e90 2023-09-29T23:31:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.746+0000 7f19e77fe700 1 -- 172.21.15.143:0/1764812773 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e800f040 con 0x7f19f00b3e90 2023-09-29T23:31:07.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.746+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1764812773 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f00a1fc0 con 0x7f19f00b3e90 2023-09-29T23:31:07.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.746+0000 7f19e77fe700 1 -- 172.21.15.143:0/1764812773 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8015440 con 0x7f19f00b3e90 2023-09-29T23:31:07.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.746+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1764812773 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f0129dd0 con 0x7f19f00b3e90 2023-09-29T23:31:07.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.746+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1764812773 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f0129ff0 con 0x7f19f00b3e90 2023-09-29T23:31:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.750+0000 7f19e77fe700 1 -- 172.21.15.143:0/1764812773 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f19e801f440 con 0x7f19f00b3e90 2023-09-29T23:31:07.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.750+0000 7f19e77fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.750+0000 7f19e77fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.750+0000 7f19e77fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.750+0000 7f19e77fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.750+0000 7f19e77fe700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f19d008e8b0 0x7f19d0090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.750+0000 7f19ef7fe700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f19d008e8b0 0x7f19d0090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:07.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.750+0000 7f19e77fe700 1 -- 172.21.15.143:0/1764812773 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f19e80b51f0 con 0x7f19f00b3e90 2023-09-29T23:31:07.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.750+0000 7f19ef7fe700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f19d008e8b0 0x7f19d0090d60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f19d80093f0 tx=0x7f19d8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:07.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.754+0000 7f19e77fe700 1 -- 172.21.15.143:0/1764812773 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f19e8085b10 con 0x7f19f00b3e90 2023-09-29T23:31:07.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1764812773 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f19f005ecb0 con 0x7f19f00b3e90 2023-09-29T23:31:07.954 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19e77fe700 1 -- 172.21.15.143:0/1764812773 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f19e801a080 con 0x7f19f00b3e90 2023-09-29T23:31:07.954 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:07.954 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:07.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 10 mgrc shutdown 2023-09-29T23:31:07.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f19d008e8b0 msgr2=0x7f19d0090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:07.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f19d008e8b0 0x7f19d0090d60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f19d80093f0 tx=0x7f19d8017000 comp rx=0 tx=0).stop 2023-09-29T23:31:07.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f19f00b3e90 msgr2=0x7f19f012a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:07.956 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f19f00b3e90 0x7f19f012a800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009fc0 tx=0x7f19e80099b0 comp rx=0 tx=0).stop 2023-09-29T23:31:07.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1764812773 shutdown_connections 2023-09-29T23:31:07.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f19d008e8b0 0x7f19d0090d60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:07.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f19f00b3e90 0x7f19f012a800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:07.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f19f009fdd0 0x7f19f012a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:07.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 --2- 172.21.15.143:0/1764812773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f0063100 0x7f19f012c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:07.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1764812773 >> 172.21.15.143:0/1764812773 conn(0x7f19f006ab30 msgr2=0x7f19f006fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:07.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1764812773 shutdown_connections 2023-09-29T23:31:07.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:07.950+0000 7f19f4ac7700 1 -- 172.21.15.143:0/1764812773 wait complete. 2023-09-29T23:31:10.969 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:11.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66f2e6700 1 Processor -- start 2023-09-29T23:31:11.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66f2e6700 1 -- start start 2023-09-29T23:31:11.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66f2e6700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66813f880 0x7ff66806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:11.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66f2e6700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff66813fd80 con 0x7ff66806b2f0 2023-09-29T23:31:11.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66f2e6700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6680b3510 con 0x7ff6680b3140 2023-09-29T23:31:11.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66f2e6700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66806b8c0 con 0x7ff66813f880 2023-09-29T23:31:11.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66c881700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66813f880 0x7ff66806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:11.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66d883700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff66806b2f0 0x7ff66806bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:36142/0 (socket says 172.21.15.143:36142) 2023-09-29T23:31:11.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66d883700 1 -- 172.21.15.143:0/1802541831 learned_addr learned my addr 172.21.15.143:0/1802541831 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:31:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff65ffff700 1 -- 172.21.15.143:0/1802541831 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660831849 0 0) 0x7ff66813fd80 con 0x7ff66806b2f0 2023-09-29T23:31:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff65ffff700 1 -- 172.21.15.143:0/1802541831 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff654003570 con 0x7ff66806b2f0 2023-09-29T23:31:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66c881700 1 -- 172.21.15.143:0/1802541831 >> v1:172.21.15.156:6789/0 conn(0x7ff6680b3140 legacy=0x7ff668132220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66c881700 1 -- 172.21.15.143:0/1802541831 >> v1:172.21.15.143:6789/0 conn(0x7ff66806b2f0 legacy=0x7ff66806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66c881700 1 -- 172.21.15.143:0/1802541831 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff668132930 con 0x7ff66813f880 2023-09-29T23:31:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff66c881700 1 --2- 172.21.15.143:0/1802541831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66813f880 0x7ff66806d8d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff664009e90 tx=0x7ff664009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3a6aef019b6166e server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:11.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff65ffff700 1 -- 172.21.15.143:0/1802541831 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66400e070 con 0x7ff66813f880 2023-09-29T23:31:11.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff65ffff700 1 -- 172.21.15.143:0/1802541831 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66400bc30 con 0x7ff66813f880 2023-09-29T23:31:11.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.078+0000 7ff65ffff700 1 -- 172.21.15.143:0/1802541831 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff664012430 con 0x7ff66813f880 2023-09-29T23:31:11.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 -- 172.21.15.143:0/1802541831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66813f880 msgr2=0x7ff66806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:11.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 --2- 172.21.15.143:0/1802541831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66813f880 0x7ff66806d8d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff664009e90 tx=0x7ff664009780 comp rx=0 tx=0).stop 2023-09-29T23:31:11.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 -- 172.21.15.143:0/1802541831 shutdown_connections 2023-09-29T23:31:11.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 --2- 172.21.15.143:0/1802541831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66813f880 0x7ff66806d8d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:11.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 -- 172.21.15.143:0/1802541831 >> 172.21.15.143:0/1802541831 conn(0x7ff668067dc0 msgr2=0x7ff6680b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:11.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 -- 172.21.15.143:0/1802541831 shutdown_connections 2023-09-29T23:31:11.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 -- 172.21.15.143:0/1802541831 wait complete. 2023-09-29T23:31:11.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 Processor -- start 2023-09-29T23:31:11.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 -- start start 2023-09-29T23:31:11.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66806b2f0 0x7ff6680ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:11.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6680b3140 0x7ff6680abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:11.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff66813f880 0x7ff6680af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:11.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680ac420 con 0x7ff66813f880 2023-09-29T23:31:11.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680ac630 con 0x7ff6680b3140 2023-09-29T23:31:11.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680ac8f0 con 0x7ff66806b2f0 2023-09-29T23:31:11.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66d082700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66806b2f0 0x7ff6680ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:11.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66d082700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66806b2f0 0x7ff6680ab9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:45324/0 (socket says 172.21.15.143:45324) 2023-09-29T23:31:11.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66d082700 1 -- 172.21.15.143:0/3636855645 learned_addr learned my addr 172.21.15.143:0/3636855645 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:11.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66d883700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff66813f880 0x7ff6680af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:11.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66c881700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6680b3140 0x7ff6680abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:11.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66d082700 1 -- 172.21.15.143:0/3636855645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6680b3140 msgr2=0x7ff6680abee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:11.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66d082700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6680b3140 0x7ff6680abee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:11.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66d082700 1 -- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff66813f880 msgr2=0x7ff6680af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:11.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66d082700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff66813f880 0x7ff6680af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:11.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66d082700 1 -- 172.21.15.143:0/3636855645 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff664009a20 con 0x7ff66806b2f0 2023-09-29T23:31:11.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66c881700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6680b3140 0x7ff6680abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:31:11.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66d082700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66806b2f0 0x7ff6680ab9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff65800c590 tx=0x7ff65800c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:11.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff65d7fa700 1 -- 172.21.15.143:0/3636855645 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658011590 con 0x7ff66806b2f0 2023-09-29T23:31:11.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff65d7fa700 1 -- 172.21.15.143:0/3636855645 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff658011710 con 0x7ff66806b2f0 2023-09-29T23:31:11.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff65d7fa700 1 -- 172.21.15.143:0/3636855645 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658011ad0 con 0x7ff66806b2f0 2023-09-29T23:31:11.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 -- 172.21.15.143:0/3636855645 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6680ad450 con 0x7ff66806b2f0 2023-09-29T23:31:11.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.082+0000 7ff66f2e6700 1 -- 172.21.15.143:0/3636855645 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff668146500 con 0x7ff66806b2f0 2023-09-29T23:31:11.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.086+0000 7ff66f2e6700 1 -- 172.21.15.143:0/3636855645 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6680ab760 con 0x7ff66806b2f0 2023-09-29T23:31:11.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.086+0000 7ff65d7fa700 1 -- 172.21.15.143:0/3636855645 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7ff65801b3b0 con 0x7ff66806b2f0 2023-09-29T23:31:11.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.086+0000 7ff65d7fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:11.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.086+0000 7ff65d7fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:11.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.086+0000 7ff65d7fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:11.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.086+0000 7ff65d7fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:11.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.086+0000 7ff65d7fa700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff64c08e8b0 0x7ff64c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:11.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.086+0000 7ff65d7fa700 1 -- 172.21.15.143:0/3636855645 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7ff6580b3dc0 con 0x7ff66806b2f0 2023-09-29T23:31:11.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.086+0000 7ff66c881700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff64c08e8b0 0x7ff64c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:11.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.090+0000 7ff66c881700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff64c08e8b0 0x7ff64c090d60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff664009e90 tx=0x7ff664019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:11.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.094+0000 7ff65d7fa700 1 -- 172.21.15.143:0/3636855645 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff658084780 con 0x7ff66806b2f0 2023-09-29T23:31:11.293 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.290+0000 7ff66f2e6700 1 -- 172.21.15.143:0/3636855645 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff66805ecb0 con 0x7ff66806b2f0 2023-09-29T23:31:11.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.290+0000 7ff65d7fa700 1 -- 172.21.15.143:0/3636855645 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7ff658017070 con 0x7ff66806b2f0 2023-09-29T23:31:11.294 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:11.294 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.290+0000 7ff66f2e6700 10 mgrc shutdown 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.290+0000 7ff66f2e6700 1 -- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff64c08e8b0 msgr2=0x7ff64c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.290+0000 7ff66f2e6700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff64c08e8b0 0x7ff64c090d60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff664009e90 tx=0x7ff664019000 comp rx=0 tx=0).stop 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.290+0000 7ff66f2e6700 1 -- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66806b2f0 msgr2=0x7ff6680ab9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.290+0000 7ff66f2e6700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66806b2f0 0x7ff6680ab9a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff65800c590 tx=0x7ff65800c8a0 comp rx=0 tx=0).stop 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.290+0000 7ff66f2e6700 1 -- 172.21.15.143:0/3636855645 shutdown_connections 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.290+0000 7ff66f2e6700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff64c08e8b0 0x7ff64c090d60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.290+0000 7ff66f2e6700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff66813f880 0x7ff6680af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.290+0000 7ff66f2e6700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6680b3140 0x7ff6680abee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.294+0000 7ff66f2e6700 1 --2- 172.21.15.143:0/3636855645 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff66806b2f0 0x7ff6680ab9a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.294+0000 7ff66f2e6700 1 -- 172.21.15.143:0/3636855645 >> 172.21.15.143:0/3636855645 conn(0x7ff668067dc0 msgr2=0x7ff668130620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:11.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.294+0000 7ff66f2e6700 1 -- 172.21.15.143:0/3636855645 shutdown_connections 2023-09-29T23:31:11.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:11.294+0000 7ff66f2e6700 1 -- 172.21.15.143:0/3636855645 wait complete. 2023-09-29T23:31:14.309 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:14.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.414+0000 7fe8bc960700 1 Processor -- start 2023-09-29T23:31:14.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.414+0000 7fe8bc960700 1 -- start start 2023-09-29T23:31:14.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8bc960700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe8b81363f0 0x7fe8b81367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:14.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8bc960700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b80a5ed0 con 0x7fe8b80a6350 2023-09-29T23:31:14.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8bc960700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b80a6720 con 0x7fe8b80a4f80 2023-09-29T23:31:14.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8bc960700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b80a5c10 con 0x7fe8b81363f0 2023-09-29T23:31:14.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8b659c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fe8b81363f0 0x7fe8b81367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:14.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8b6d9d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe8b80a6350 0x7fe8b806b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42020/0 (socket says 172.21.15.143:42020) 2023-09-29T23:31:14.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8b6d9d700 1 -- 172.21.15.143:0/2156404184 learned_addr learned my addr 172.21.15.143:0/2156404184 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:31:14.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8b659c700 1 -- 172.21.15.143:0/2156404184 >> v1:172.21.15.156:6789/0 conn(0x7fe8b80a4f80 legacy=0x7fe8b80a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:14.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8b659c700 1 -- 172.21.15.143:0/2156404184 >> v1:172.21.15.143:6789/0 conn(0x7fe8b80a6350 legacy=0x7fe8b806b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:14.422 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8b5d9b700 1 -- 172.21.15.143:0/2156404184 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006348083 0 0) 0x7fe8b80a5ed0 con 0x7fe8b80a6350 2023-09-29T23:31:14.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8b659c700 1 -- 172.21.15.143:0/2156404184 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b806ba70 con 0x7fe8b81363f0 2023-09-29T23:31:14.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8b659c700 1 --2- 172.21.15.143:0/2156404184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b81363f0 0x7fe8b81367c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0098e0 tx=0x7fe8ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa56e065b1f7e5d server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:14.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8b5d9b700 1 -- 172.21.15.143:0/2156404184 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac009e80 con 0x7fe8b81363f0 2023-09-29T23:31:14.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8b5d9b700 1 -- 172.21.15.143:0/2156404184 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ac00b860 con 0x7fe8b81363f0 2023-09-29T23:31:14.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8b5d9b700 1 -- 172.21.15.143:0/2156404184 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00bbe0 con 0x7fe8b81363f0 2023-09-29T23:31:14.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8bc960700 1 -- 172.21.15.143:0/2156404184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b81363f0 msgr2=0x7fe8b81367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:14.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8bc960700 1 --2- 172.21.15.143:0/2156404184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b81363f0 0x7fe8b81367c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0098e0 tx=0x7fe8ac0091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:14.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8bc960700 1 -- 172.21.15.143:0/2156404184 shutdown_connections 2023-09-29T23:31:14.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8bc960700 1 --2- 172.21.15.143:0/2156404184 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b81363f0 0x7fe8b81367c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:14.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8bc960700 1 -- 172.21.15.143:0/2156404184 >> 172.21.15.143:0/2156404184 conn(0x7fe8b80b6930 msgr2=0x7fe8b80b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:14.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8bc960700 1 -- 172.21.15.143:0/2156404184 shutdown_connections 2023-09-29T23:31:14.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.418+0000 7fe8bc960700 1 -- 172.21.15.143:0/2156404184 wait complete. 2023-09-29T23:31:14.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8bc960700 1 Processor -- start 2023-09-29T23:31:14.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8bc960700 1 -- start start 2023-09-29T23:31:14.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8bc960700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b80a4f80 0x7fe8b80a8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:14.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8bc960700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe8b80a6350 0x7fe8b81433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8bc960700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b81363f0 0x7fe8b8143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8bc960700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81316b0 con 0x7fe8b80a4f80 2023-09-29T23:31:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8bc960700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b81533b0 con 0x7fe8b81363f0 2023-09-29T23:31:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8bc960700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b8133df0 con 0x7fe8b80a6350 2023-09-29T23:31:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8b6d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b80a4f80 0x7fe8b80a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8b659c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe8b80a6350 0x7fe8b81433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8b6d9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b80a4f80 0x7fe8b80a8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42814/0 (socket says 172.21.15.143:42814) 2023-09-29T23:31:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8b6d9d700 1 -- 172.21.15.143:0/4080128494 learned_addr learned my addr 172.21.15.143:0/4080128494 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8b759e700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b81363f0 0x7fe8b8143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8b6d9d700 1 -- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe8b80a6350 msgr2=0x7fe8b81433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:14.427 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8b6d9d700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe8b80a6350 0x7fe8b81433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:14.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8b6d9d700 1 -- 172.21.15.143:0/4080128494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b81363f0 msgr2=0x7fe8b8143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:14.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8b6d9d700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b81363f0 0x7fe8b8143920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:14.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8b6d9d700 1 -- 172.21.15.143:0/4080128494 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac009420 con 0x7fe8b80a4f80 2023-09-29T23:31:14.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8b6d9d700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b80a4f80 0x7fe8b80a8510 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c810 tx=0x7fe8a8007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:14.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe89effd700 1 -- 172.21.15.143:0/4080128494 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8005c00 con 0x7fe8b80a4f80 2023-09-29T23:31:14.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe89effd700 1 -- 172.21.15.143:0/4080128494 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a8005d60 con 0x7fe8b80a4f80 2023-09-29T23:31:14.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8bc960700 1 -- 172.21.15.143:0/4080128494 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b8131010 con 0x7fe8b80a4f80 2023-09-29T23:31:14.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe89effd700 1 -- 172.21.15.143:0/4080128494 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8007320 con 0x7fe8b80a4f80 2023-09-29T23:31:14.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.422+0000 7fe8bc960700 1 -- 172.21.15.143:0/4080128494 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b8134c60 con 0x7fe8b80a4f80 2023-09-29T23:31:14.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.426+0000 7fe8bc960700 1 -- 172.21.15.143:0/4080128494 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b805ecb0 con 0x7fe8b80a4f80 2023-09-29T23:31:14.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.426+0000 7fe89effd700 1 -- 172.21.15.143:0/4080128494 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fe8a801b3b0 con 0x7fe8b80a4f80 2023-09-29T23:31:14.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.426+0000 7fe89effd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:14.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.430+0000 7fe89effd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:14.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.430+0000 7fe89effd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:14.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.430+0000 7fe89effd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:14.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.430+0000 7fe89effd700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe8a408e830 0x7fe8a4090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:14.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.430+0000 7fe8b659c700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe8a408e830 0x7fe8a4090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:14.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.430+0000 7fe89effd700 1 -- 172.21.15.143:0/4080128494 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fe8a80b3d60 con 0x7fe8b80a4f80 2023-09-29T23:31:14.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.430+0000 7fe8b659c700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe8a408e830 0x7fe8a4090ce0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0093f0 tx=0x7fe8ac017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:14.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.438+0000 7fe89effd700 1 -- 172.21.15.143:0/4080128494 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe8a80847b0 con 0x7fe8b80a4f80 2023-09-29T23:31:14.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 -- 172.21.15.143:0/4080128494 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe8b8133f30 con 0x7fe8b80a4f80 2023-09-29T23:31:14.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe89effd700 1 -- 172.21.15.143:0/4080128494 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fe8b8133f30 con 0x7fe8b80a4f80 2023-09-29T23:31:14.712 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:14.712 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:14.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 10 mgrc shutdown 2023-09-29T23:31:14.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 -- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe8a408e830 msgr2=0x7fe8a4090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:14.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe8a408e830 0x7fe8a4090ce0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe8ac0093f0 tx=0x7fe8ac017000 comp rx=0 tx=0).stop 2023-09-29T23:31:14.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 -- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b80a4f80 msgr2=0x7fe8b80a8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:14.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b80a4f80 0x7fe8b80a8510 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800c810 tx=0x7fe8a8007d90 comp rx=0 tx=0).stop 2023-09-29T23:31:14.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 -- 172.21.15.143:0/4080128494 shutdown_connections 2023-09-29T23:31:14.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe8a408e830 0x7fe8a4090ce0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:14.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8b81363f0 0x7fe8b8143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:14.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe8b80a6350 0x7fe8b81433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:14.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 --2- 172.21.15.143:0/4080128494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8b80a4f80 0x7fe8b80a8510 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:14.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 -- 172.21.15.143:0/4080128494 >> 172.21.15.143:0/4080128494 conn(0x7fe8b80b6930 msgr2=0x7fe8b8144670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 -- 172.21.15.143:0/4080128494 shutdown_connections 2023-09-29T23:31:14.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:14.634+0000 7fe8bc960700 1 -- 172.21.15.143:0/4080128494 wait complete. 2023-09-29T23:31:17.715 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:17.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10123c4700 1 Processor -- start 2023-09-29T23:31:17.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10123c4700 1 -- start start 2023-09-29T23:31:17.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10123c4700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f100c13d850 0x7f100c1495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:17.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10123c4700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100c09ff40 con 0x7f100c066ec0 2023-09-29T23:31:17.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10123c4700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100c067290 con 0x7f100c0a1170 2023-09-29T23:31:17.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10123c4700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100c0a0510 con 0x7f100c13d850 2023-09-29T23:31:17.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10113c2700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f100c066ec0 0x7f100c09f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:42026/0 (socket says 172.21.15.143:42026) 2023-09-29T23:31:17.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10113c2700 1 -- 172.21.15.143:0/542916074 learned_addr learned my addr 172.21.15.143:0/542916074 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:31:17.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1011bc3700 1 --2- 172.21.15.143:0/542916074 >> v2:172.21.15.156:3300/0 conn(0x7f100c13d850 0x7f100c1495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:17.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196588863 0 0) 0x7f100c09ff40 con 0x7f100c066ec0 2023-09-29T23:31:17.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff8003570 con 0x7f100c066ec0 2023-09-29T23:31:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047444517 0 0) 0x7f100c067290 con 0x7f100c0a1170 2023-09-29T23:31:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f100c09ff40 con 0x7f100c0a1170 2023-09-29T23:31:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4095340957 0 0) 0x7f0ff8003570 con 0x7f100c066ec0 2023-09-29T23:31:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 >> v2:172.21.15.156:3300/0 conn(0x7f100c13d850 msgr2=0x7f100c1495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 --2- 172.21.15.143:0/542916074 >> v2:172.21.15.156:3300/0 conn(0x7f100c13d850 0x7f100c1495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 >> v1:172.21.15.156:6789/0 conn(0x7f100c0a1170 legacy=0x7f100c13d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c06e400 con 0x7f100c066ec0 2023-09-29T23:31:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251153714 0 0) 0x7f0ffc001470 con 0x7f100c066ec0 2023-09-29T23:31:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 >> v1:172.21.15.143:6789/0 conn(0x7f100c066ec0 legacy=0x7f100c09f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 --2- 172.21.15.143:0/542916074 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0ff8003bf0 0x7f0ff8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:17.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 --2- 172.21.15.143:0/542916074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ff8004810 0x7f0ff8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 --2- 172.21.15.143:0/542916074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff80051c0 0x7f0ff800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0a0510 con 0x7f0ff8004810 2023-09-29T23:31:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff800c160 con 0x7f0ff80051c0 2023-09-29T23:31:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff800c320 con 0x7f0ff8003bf0 2023-09-29T23:31:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10113c2700 1 --2- 172.21.15.143:0/542916074 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0ff8003bf0 0x7f0ff8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1010bc1700 1 --2- 172.21.15.143:0/542916074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ff8004810 0x7f0ff8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f1011bc3700 1 --2- 172.21.15.143:0/542916074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff80051c0 0x7f0ff800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10113c2700 1 -- 172.21.15.143:0/542916074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff80051c0 msgr2=0x7f0ff800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10113c2700 1 --2- 172.21.15.143:0/542916074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff80051c0 0x7f0ff800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10113c2700 1 -- 172.21.15.143:0/542916074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ff8004810 msgr2=0x7f0ff8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:17.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.826+0000 7f10113c2700 1 --2- 172.21.15.143:0/542916074 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ff8004810 0x7f0ff8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:17.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f1011bc3700 1 -- 172.21.15.143:0/542916074 reap_dead start 2023-09-29T23:31:17.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f10113c2700 1 -- 172.21.15.143:0/542916074 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff8003570 con 0x7f0ff8003bf0 2023-09-29T23:31:17.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f10113c2700 1 --2- 172.21.15.143:0/542916074 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0ff8003bf0 0x7f0ff8003fe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c2a0 tx=0x7f0ffc001e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc718dba345c87f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:17.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc00cb90 con 0x7f0ff8003bf0 2023-09-29T23:31:17.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc00ccf0 con 0x7f0ff8003bf0 2023-09-29T23:31:17.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f1003fff700 1 -- 172.21.15.143:0/542916074 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc005b70 con 0x7f0ff8003bf0 2023-09-29T23:31:17.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f10123c4700 1 -- 172.21.15.143:0/542916074 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0ff8003bf0 msgr2=0x7f0ff8003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:17.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f10123c4700 1 --2- 172.21.15.143:0/542916074 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0ff8003bf0 0x7f0ff8003fe0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c2a0 tx=0x7f0ffc001e10 comp rx=0 tx=0).stop 2023-09-29T23:31:17.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f10123c4700 1 -- 172.21.15.143:0/542916074 shutdown_connections 2023-09-29T23:31:17.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f10123c4700 1 --2- 172.21.15.143:0/542916074 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0ff8003bf0 0x7f0ff8003fe0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:17.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f10123c4700 1 -- 172.21.15.143:0/542916074 >> 172.21.15.143:0/542916074 conn(0x7f100c06f200 msgr2=0x7f100c069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:17.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f10123c4700 1 -- 172.21.15.143:0/542916074 shutdown_connections 2023-09-29T23:31:17.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.830+0000 7f10123c4700 1 -- 172.21.15.143:0/542916074 wait complete. 2023-09-29T23:31:17.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10123c4700 1 Processor -- start 2023-09-29T23:31:17.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10123c4700 1 -- start start 2023-09-29T23:31:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10123c4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8003bf0 0x7f100c141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10123c4700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f100c09a690 0x7f100c1513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10123c4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f100c151920 0x7f100c143570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10123c4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c150f50 con 0x7f100c151920 2023-09-29T23:31:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10123c4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c097970 con 0x7f0ff8003bf0 2023-09-29T23:31:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10123c4700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c097c30 con 0x7f100c09a690 2023-09-29T23:31:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f1010bc1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f100c09a690 0x7f100c1513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f1011bc3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f100c151920 0x7f100c143570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f1010bc1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f100c09a690 0x7f100c1513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:41854/0 (socket says 172.21.15.143:41854) 2023-09-29T23:31:17.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f1010bc1700 1 -- 172.21.15.143:0/1568511858 learned_addr learned my addr 172.21.15.143:0/1568511858 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10113c2700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8003bf0 0x7f100c141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f1011bc3700 1 -- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f100c09a690 msgr2=0x7f100c1513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f1011bc3700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f100c09a690 0x7f100c1513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f1011bc3700 1 -- 172.21.15.143:0/1568511858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8003bf0 msgr2=0x7f100c141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f1011bc3700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8003bf0 0x7f100c141950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f1011bc3700 1 -- 172.21.15.143:0/1568511858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c151d70 con 0x7f100c151920 2023-09-29T23:31:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f1010bc1700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f100c09a690 0x7f100c1513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f1011bc3700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f100c151920 0x7f100c143570 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f100c0a0bc0 tx=0x7f100400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10017fa700 1 -- 172.21.15.143:0/1568511858 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100400e040 con 0x7f100c151920 2023-09-29T23:31:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10017fa700 1 -- 172.21.15.143:0/1568511858 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100400f040 con 0x7f100c151920 2023-09-29T23:31:17.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10123c4700 1 -- 172.21.15.143:0/1568511858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c09aaa0 con 0x7f100c151920 2023-09-29T23:31:17.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10017fa700 1 -- 172.21.15.143:0/1568511858 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100400fd70 con 0x7f100c151920 2023-09-29T23:31:17.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.834+0000 7f10123c4700 1 -- 172.21.15.143:0/1568511858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c096de0 con 0x7f100c151920 2023-09-29T23:31:17.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.838+0000 7f10123c4700 1 -- 172.21.15.143:0/1568511858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c05ecb0 con 0x7f100c151920 2023-09-29T23:31:17.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.838+0000 7f10017fa700 1 -- 172.21.15.143:0/1568511858 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f1004015640 con 0x7f100c151920 2023-09-29T23:31:17.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.838+0000 7f10017fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.838+0000 7f10017fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.838+0000 7f10017fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.838+0000 7f10017fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.838+0000 7f10017fa700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ff008e8b0 0x7f0ff0090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.838+0000 7f10113c2700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ff008e8b0 0x7f0ff0090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:17.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.838+0000 7f10017fa700 1 -- 172.21.15.143:0/1568511858 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f10040b5ab0 con 0x7f100c151920 2023-09-29T23:31:17.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.842+0000 7f10113c2700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ff008e8b0 0x7f0ff0090d60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc008fd0 tx=0x7f0ffc015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:17.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:17.846+0000 7f10017fa700 1 -- 172.21.15.143:0/1568511858 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f1004086500 con 0x7f100c151920 2023-09-29T23:31:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.034+0000 7f10123c4700 1 -- 172.21.15.143:0/1568511858 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f100c06e6c0 con 0x7f100c151920 2023-09-29T23:31:18.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.034+0000 7f10017fa700 1 -- 172.21.15.143:0/1568511858 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f1004089b20 con 0x7f100c151920 2023-09-29T23:31:18.040 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:18.040 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.038+0000 7f10123c4700 10 mgrc shutdown 2023-09-29T23:31:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.038+0000 7f10123c4700 1 -- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ff008e8b0 msgr2=0x7f0ff0090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.038+0000 7f10123c4700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ff008e8b0 0x7f0ff0090d60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc008fd0 tx=0x7f0ffc015000 comp rx=0 tx=0).stop 2023-09-29T23:31:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.038+0000 7f10123c4700 1 -- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f100c151920 msgr2=0x7f100c143570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.038+0000 7f10123c4700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f100c151920 0x7f100c143570 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f100c0a0bc0 tx=0x7f100400b040 comp rx=0 tx=0).stop 2023-09-29T23:31:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.042+0000 7f10123c4700 1 -- 172.21.15.143:0/1568511858 shutdown_connections 2023-09-29T23:31:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.042+0000 7f10123c4700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0ff008e8b0 0x7f0ff0090d60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:18.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.042+0000 7f10123c4700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f100c151920 0x7f100c143570 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.042+0000 7f10123c4700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f100c09a690 0x7f100c1513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.042+0000 7f10123c4700 1 --2- 172.21.15.143:0/1568511858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8003bf0 0x7f100c141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.042+0000 7f10123c4700 1 -- 172.21.15.143:0/1568511858 >> 172.21.15.143:0/1568511858 conn(0x7f100c06f200 msgr2=0x7f100c1394b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.042+0000 7f10123c4700 1 -- 172.21.15.143:0/1568511858 shutdown_connections 2023-09-29T23:31:18.046 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:18.042+0000 7f10123c4700 1 -- 172.21.15.143:0/1568511858 wait complete. 2023-09-29T23:31:21.060 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:21.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 Processor -- start 2023-09-29T23:31:21.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 -- start start 2023-09-29T23:31:21.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ffbb4066ec0 0x7ffbb406c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffbb413c010 0x7ffbb413c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb406cd10 con 0x7ffbb406e930 2023-09-29T23:31:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb40a2550 con 0x7ffbb413c010 2023-09-29T23:31:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb4067290 con 0x7ffbb4066ec0 2023-09-29T23:31:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbb9eed700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffbb413c010 0x7ffbb413c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbb9eed700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffbb413c010 0x7ffbb413c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:42852/0 (socket says 172.21.15.143:42852) 2023-09-29T23:31:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbb9eed700 1 -- 172.21.15.143:0/3801263777 learned_addr learned my addr 172.21.15.143:0/3801263777 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:21.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbb96ec700 1 --2- 172.21.15.143:0/3801263777 >> v2:172.21.15.156:3300/0 conn(0x7ffbb4066ec0 0x7ffbb406c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:21.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbb9eed700 1 -- 172.21.15.143:0/3801263777 >> v2:172.21.15.156:3300/0 conn(0x7ffbb4066ec0 msgr2=0x7ffbb406c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:21.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbb9eed700 1 --2- 172.21.15.143:0/3801263777 >> v2:172.21.15.156:3300/0 conn(0x7ffbb4066ec0 0x7ffbb406c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:21.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbb9eed700 1 -- 172.21.15.143:0/3801263777 >> v1:172.21.15.156:6789/0 conn(0x7ffbb406e930 legacy=0x7ffbb40a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:21.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbb9eed700 1 -- 172.21.15.143:0/3801263777 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb406ad80 con 0x7ffbb413c010 2023-09-29T23:31:21.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbb9eed700 1 --2- 172.21.15.143:0/3801263777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbb413c010 0x7ffbb413c490 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40a4fb0 tx=0x7ffbac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf0eb53c6107208 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:21.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbabfff700 1 -- 172.21.15.143:0/3801263777 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00e070 con 0x7ffbb413c010 2023-09-29T23:31:21.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbabfff700 1 -- 172.21.15.143:0/3801263777 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac00ba20 con 0x7ffbb413c010 2023-09-29T23:31:21.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbabfff700 1 -- 172.21.15.143:0/3801263777 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac0092e0 con 0x7ffbb413c010 2023-09-29T23:31:21.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 -- 172.21.15.143:0/3801263777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbb413c010 msgr2=0x7ffbb413c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:21.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 --2- 172.21.15.143:0/3801263777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbb413c010 0x7ffbb413c490 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ffbb40a4fb0 tx=0x7ffbac009840 comp rx=0 tx=0).stop 2023-09-29T23:31:21.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 -- 172.21.15.143:0/3801263777 shutdown_connections 2023-09-29T23:31:21.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 --2- 172.21.15.143:0/3801263777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbb413c010 0x7ffbb413c490 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:21.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 --2- 172.21.15.143:0/3801263777 >> v2:172.21.15.156:3300/0 conn(0x7ffbb4066ec0 0x7ffbb406c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:21.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 -- 172.21.15.143:0/3801263777 >> 172.21.15.143:0/3801263777 conn(0x7ffbb406f200 msgr2=0x7ffbb4069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:21.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 -- 172.21.15.143:0/3801263777 shutdown_connections 2023-09-29T23:31:21.178 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.170+0000 7ffbba6ee700 1 -- 172.21.15.143:0/3801263777 wait complete. 2023-09-29T23:31:21.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbba6ee700 1 Processor -- start 2023-09-29T23:31:21.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbba6ee700 1 -- start start 2023-09-29T23:31:21.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbba6ee700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffbb4066ec0 0x7ffbb4099a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:21.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbba6ee700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbb406e930 0x7ffbb4098900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:21.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbba6ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb413c010 0x7ffbb4095a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:21.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbba6ee700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40962e0 con 0x7ffbb406e930 2023-09-29T23:31:21.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbba6ee700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb40965a0 con 0x7ffbb413c010 2023-09-29T23:31:21.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbba6ee700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb4095f80 con 0x7ffbb4066ec0 2023-09-29T23:31:21.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb96ec700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffbb4066ec0 0x7ffbb4099a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:21.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb8eeb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbb406e930 0x7ffbb4098900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:21.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb96ec700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffbb4066ec0 0x7ffbb4099a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:41868/0 (socket says 172.21.15.143:41868) 2023-09-29T23:31:21.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb96ec700 1 -- 172.21.15.143:0/1506627227 learned_addr learned my addr 172.21.15.143:0/1506627227 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:21.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb9eed700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb413c010 0x7ffbb4095a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb96ec700 1 -- 172.21.15.143:0/1506627227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb413c010 msgr2=0x7ffbb4095a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb96ec700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb413c010 0x7ffbb4095a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb96ec700 1 -- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbb406e930 msgr2=0x7ffbb4098900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb96ec700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbb406e930 0x7ffbb4098900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb96ec700 1 -- 172.21.15.143:0/1506627227 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac009ae0 con 0x7ffbb4066ec0 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb8eeb700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbb406e930 0x7ffbb4098900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbb96ec700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffbb4066ec0 0x7ffbb4099a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000ec80 tx=0x7ffbb00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffba97fa700 1 -- 172.21.15.143:0/1506627227 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000c8e0 con 0x7ffbb4066ec0 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffba97fa700 1 -- 172.21.15.143:0/1506627227 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb000ca40 con 0x7ffbb4066ec0 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbba6ee700 1 -- 172.21.15.143:0/1506627227 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb4140b50 con 0x7ffbb4066ec0 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffba97fa700 1 -- 172.21.15.143:0/1506627227 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000cde0 con 0x7ffbb4066ec0 2023-09-29T23:31:21.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.174+0000 7ffbba6ee700 1 -- 172.21.15.143:0/1506627227 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb409a230 con 0x7ffbb4066ec0 2023-09-29T23:31:21.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.178+0000 7ffbba6ee700 1 -- 172.21.15.143:0/1506627227 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb405ecb0 con 0x7ffbb4066ec0 2023-09-29T23:31:21.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.178+0000 7ffba97fa700 1 -- 172.21.15.143:0/1506627227 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7ffbb0013680 con 0x7ffbb4066ec0 2023-09-29T23:31:21.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.178+0000 7ffba97fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:21.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.178+0000 7ffba97fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:21.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.178+0000 7ffba97fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:21.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.178+0000 7ffba97fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:21.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.178+0000 7ffba97fa700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ffb9808e8b0 0x7ffb98090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:21.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.178+0000 7ffbb8eeb700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ffb9808e8b0 0x7ffb98090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:21.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.178+0000 7ffba97fa700 1 -- 172.21.15.143:0/1506627227 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7ffbb00b4f90 con 0x7ffbb4066ec0 2023-09-29T23:31:21.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.182+0000 7ffbb8eeb700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ffb9808e8b0 0x7ffb98090d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffbb406d660 tx=0x7ffba4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:21.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.186+0000 7ffba97fa700 1 -- 172.21.15.143:0/1506627227 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ffbb00858b0 con 0x7ffbb4066ec0 2023-09-29T23:31:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.378+0000 7ffbba6ee700 1 -- 172.21.15.143:0/1506627227 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ffbb406b040 con 0x7ffbb4066ec0 2023-09-29T23:31:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.378+0000 7ffba97fa700 1 -- 172.21.15.143:0/1506627227 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7ffbb0021100 con 0x7ffbb4066ec0 2023-09-29T23:31:21.383 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:21.384 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 10 mgrc shutdown 2023-09-29T23:31:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 -- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ffb9808e8b0 msgr2=0x7ffb98090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ffb9808e8b0 0x7ffb98090d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffbb406d660 tx=0x7ffba4009040 comp rx=0 tx=0).stop 2023-09-29T23:31:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 -- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffbb4066ec0 msgr2=0x7ffbb4099a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffbb4066ec0 0x7ffbb4099a10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000ec80 tx=0x7ffbb00091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 -- 172.21.15.143:0/1506627227 shutdown_connections 2023-09-29T23:31:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ffb9808e8b0 0x7ffb98090d60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:21.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb413c010 0x7ffbb4095a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:21.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffbb406e930 0x7ffbb4098900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:21.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 --2- 172.21.15.143:0/1506627227 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffbb4066ec0 0x7ffbb4099a10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:21.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 -- 172.21.15.143:0/1506627227 >> 172.21.15.143:0/1506627227 conn(0x7ffbb406f200 msgr2=0x7ffbb406fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:21.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 -- 172.21.15.143:0/1506627227 shutdown_connections 2023-09-29T23:31:21.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:21.382+0000 7ffbba6ee700 1 -- 172.21.15.143:0/1506627227 wait complete. 2023-09-29T23:31:24.398 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:24.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6f335700 1 Processor -- start 2023-09-29T23:31:24.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6f335700 1 -- start start 2023-09-29T23:31:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6f335700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6813fce0 0x7f0e680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6f335700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e680a48e0 con 0x7f0e680a5310 2023-09-29T23:31:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6f335700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e680b7640 con 0x7f0e680b71e0 2023-09-29T23:31:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6f335700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e680a3de0 con 0x7f0e6813fce0 2023-09-29T23:31:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6e333700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0e680a5310 0x7f0e6813f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33616/0 (socket says 172.21.15.143:33616) 2023-09-29T23:31:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6e333700 1 -- 172.21.15.143:0/2712570034 learned_addr learned my addr 172.21.15.143:0/2712570034 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:31:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6db32700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6813fce0 0x7f0e680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1073359133 0 0) 0x7f0e680a48e0 con 0x7f0e680a5310 2023-09-29T23:31:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e54003570 con 0x7f0e680a5310 2023-09-29T23:31:24.512 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2506026234 0 0) 0x7f0e54003570 con 0x7f0e680a5310 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6813fce0 msgr2=0x7f0e680a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 --2- 172.21.15.143:0/2712570034 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6813fce0 0x7f0e680a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 >> v1:172.21.15.156:6789/0 conn(0x7f0e680b71e0 legacy=0x7f0e6814aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e680a40a0 con 0x7f0e680a5310 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251153714 0 0) 0x7f0e64001490 con 0x7f0e680a5310 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 >> v1:172.21.15.143:6789/0 conn(0x7f0e680a5310 legacy=0x7f0e6813f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 --2- 172.21.15.143:0/2712570034 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6813fce0 0x7f0e54003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 --2- 172.21.15.143:0/2712570034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e54003ff0 0x7f0e54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 --2- 172.21.15.143:0/2712570034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e54004d50 0x7f0e540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e680a3de0 con 0x7f0e54004d50 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5400bb70 con 0x7f0e54003ff0 2023-09-29T23:31:24.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5400bd30 con 0x7f0e6813fce0 2023-09-29T23:31:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6e333700 1 --2- 172.21.15.143:0/2712570034 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6813fce0 0x7f0e54003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6eb34700 1 --2- 172.21.15.143:0/2712570034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e54004d50 0x7f0e540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6db32700 1 --2- 172.21.15.143:0/2712570034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e54003ff0 0x7f0e54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6eb34700 1 -- 172.21.15.143:0/2712570034 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6813fce0 msgr2=0x7f0e54003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6eb34700 1 --2- 172.21.15.143:0/2712570034 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6813fce0 0x7f0e54003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6eb34700 1 -- 172.21.15.143:0/2712570034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e54003ff0 msgr2=0x7f0e54004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6eb34700 1 --2- 172.21.15.143:0/2712570034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e54003ff0 0x7f0e54004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6eb34700 1 -- 172.21.15.143:0/2712570034 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e54003570 con 0x7f0e54004d50 2023-09-29T23:31:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6eb34700 1 --2- 172.21.15.143:0/2712570034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e54004d50 0x7f0e540051c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0e600020a0 tx=0x7f0e600016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82560a323418d9b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600019e0 con 0x7f0e54004d50 2023-09-29T23:31:24.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6000c8b0 con 0x7f0e54004d50 2023-09-29T23:31:24.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.506+0000 7f0e6d331700 1 -- 172.21.15.143:0/2712570034 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6000cc50 con 0x7f0e54004d50 2023-09-29T23:31:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 -- 172.21.15.143:0/2712570034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e54004d50 msgr2=0x7f0e540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 --2- 172.21.15.143:0/2712570034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e54004d50 0x7f0e540051c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0e600020a0 tx=0x7f0e600016d0 comp rx=0 tx=0).stop 2023-09-29T23:31:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6eb34700 1 -- 172.21.15.143:0/2712570034 reap_dead start 2023-09-29T23:31:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 -- 172.21.15.143:0/2712570034 shutdown_connections 2023-09-29T23:31:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 -- 172.21.15.143:0/2712570034 >> 172.21.15.143:0/2712570034 conn(0x7f0e68066c90 msgr2=0x7f0e68067090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 -- 172.21.15.143:0/2712570034 shutdown_connections 2023-09-29T23:31:24.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 -- 172.21.15.143:0/2712570034 wait complete. 2023-09-29T23:31:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 Processor -- start 2023-09-29T23:31:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 -- start start 2023-09-29T23:31:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e68143810 0x7f0e6814ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6809a320 0x7f0e6814f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e680a2c60 0x7f0e6806aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6806de30 con 0x7f0e680a2c60 2023-09-29T23:31:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e68143be0 con 0x7f0e68143810 2023-09-29T23:31:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6f335700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6806b770 con 0x7f0e6809a320 2023-09-29T23:31:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6eb34700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e680a2c60 0x7f0e6806aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6db32700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6809a320 0x7f0e6814f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:24.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6eb34700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e680a2c60 0x7f0e6806aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54762/0 (socket says 172.21.15.143:54762) 2023-09-29T23:31:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6eb34700 1 -- 172.21.15.143:0/489465183 learned_addr learned my addr 172.21.15.143:0/489465183 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6e333700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e68143810 0x7f0e6814ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6eb34700 1 -- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6809a320 msgr2=0x7f0e6814f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6eb34700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6809a320 0x7f0e6814f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6eb34700 1 -- 172.21.15.143:0/489465183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e68143810 msgr2=0x7f0e6814ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6eb34700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e68143810 0x7f0e6814ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6eb34700 1 -- 172.21.15.143:0/489465183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6806ba30 con 0x7f0e680a2c60 2023-09-29T23:31:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.510+0000 7f0e6eb34700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e680a2c60 0x7f0e6806aae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400bb40 tx=0x7f0e60016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.514+0000 7f0e5e7fc700 1 -- 172.21.15.143:0/489465183 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600019e0 con 0x7f0e680a2c60 2023-09-29T23:31:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.514+0000 7f0e5e7fc700 1 -- 172.21.15.143:0/489465183 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6000ced0 con 0x7f0e680a2c60 2023-09-29T23:31:24.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.514+0000 7f0e5e7fc700 1 -- 172.21.15.143:0/489465183 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60012a50 con 0x7f0e680a2c60 2023-09-29T23:31:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.514+0000 7f0e6f335700 1 -- 172.21.15.143:0/489465183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e681440f0 con 0x7f0e680a2c60 2023-09-29T23:31:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.514+0000 7f0e6f335700 1 -- 172.21.15.143:0/489465183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6806b240 con 0x7f0e680a2c60 2023-09-29T23:31:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.514+0000 7f0e6f335700 1 -- 172.21.15.143:0/489465183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6809a6f0 con 0x7f0e680a2c60 2023-09-29T23:31:24.520 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.514+0000 7f0e5e7fc700 1 -- 172.21.15.143:0/489465183 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f0e60029460 con 0x7f0e680a2c60 2023-09-29T23:31:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.514+0000 7f0e5e7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.518+0000 7f0e5e7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.518+0000 7f0e5e7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.518+0000 7f0e5e7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.518+0000 7f0e5e7fc700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e4c08e8e0 0x7f0e4c090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.518+0000 7f0e6e333700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e4c08e8e0 0x7f0e4c090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.518+0000 7f0e5e7fc700 1 -- 172.21.15.143:0/489465183 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f0e600b6310 con 0x7f0e680a2c60 2023-09-29T23:31:24.521 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.518+0000 7f0e6e333700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e4c08e8e0 0x7f0e4c090d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e64001d20 tx=0x7f0e64001e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:24.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.522+0000 7f0e5e7fc700 1 -- 172.21.15.143:0/489465183 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0e6809a6f0 con 0x7f0e680a2c60 2023-09-29T23:31:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.714+0000 7f0e6f335700 1 -- 172.21.15.143:0/489465183 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0e6809a6f0 con 0x7f0e680a2c60 2023-09-29T23:31:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e5e7fc700 1 -- 172.21.15.143:0/489465183 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f0e6809a6f0 con 0x7f0e680a2c60 2023-09-29T23:31:24.721 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:24.721 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:24.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 10 mgrc shutdown 2023-09-29T23:31:24.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 -- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e4c08e8e0 msgr2=0x7f0e4c090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:24.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e4c08e8e0 0x7f0e4c090d90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0e64001d20 tx=0x7f0e64001e90 comp rx=0 tx=0).stop 2023-09-29T23:31:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 -- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e680a2c60 msgr2=0x7f0e6806aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e680a2c60 0x7f0e6806aae0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0e5400bb40 tx=0x7f0e60016000 comp rx=0 tx=0).stop 2023-09-29T23:31:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 -- 172.21.15.143:0/489465183 shutdown_connections 2023-09-29T23:31:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e4c08e8e0 0x7f0e4c090d90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e680a2c60 0x7f0e6806aae0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e6809a320 0x7f0e6814f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 --2- 172.21.15.143:0/489465183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e68143810 0x7f0e6814ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 -- 172.21.15.143:0/489465183 >> 172.21.15.143:0/489465183 conn(0x7f0e68066c90 msgr2=0x7f0e680a7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 -- 172.21.15.143:0/489465183 shutdown_connections 2023-09-29T23:31:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:24.718+0000 7f0e6f335700 1 -- 172.21.15.143:0/489465183 wait complete. 2023-09-29T23:31:26.027 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:31:26.030 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:31:27.736 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:27.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38db07a700 1 Processor -- start 2023-09-29T23:31:27.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38db07a700 1 -- start start 2023-09-29T23:31:27.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38db07a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f38d40a3770 0x7f38d40b71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:27.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38db07a700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d40a48e0 con 0x7f38d413fce0 2023-09-29T23:31:27.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38db07a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d40a3b60 con 0x7f38d40a5310 2023-09-29T23:31:27.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38db07a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d40a3de0 con 0x7f38d40a3770 2023-09-29T23:31:27.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38d9877700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f38d413fce0 0x7f38d4148a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33618/0 (socket says 172.21.15.143:33618) 2023-09-29T23:31:27.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38d9877700 1 -- 172.21.15.143:0/767894832 learned_addr learned my addr 172.21.15.143:0/767894832 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:31:27.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38da879700 1 --2- 172.21.15.143:0/767894832 >> v2:172.21.15.143:3300/0 conn(0x7f38d40a3770 0x7f38d40b71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:27.855 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38d9076700 1 -- 172.21.15.143:0/767894832 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075748014 0 0) 0x7f38d40a48e0 con 0x7f38d413fce0 2023-09-29T23:31:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38d9076700 1 -- 172.21.15.143:0/767894832 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38bc003570 con 0x7f38d413fce0 2023-09-29T23:31:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38d9076700 1 -- 172.21.15.143:0/767894832 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000888695 0 0) 0x7f38d40a3b60 con 0x7f38d40a5310 2023-09-29T23:31:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38d9076700 1 -- 172.21.15.143:0/767894832 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d40a48e0 con 0x7f38d40a5310 2023-09-29T23:31:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38d9076700 1 -- 172.21.15.143:0/767894832 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1256373172 0 0) 0x7f38bc003570 con 0x7f38d413fce0 2023-09-29T23:31:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38da879700 1 -- 172.21.15.143:0/767894832 >> v1:172.21.15.156:6789/0 conn(0x7f38d40a5310 legacy=0x7f38d413f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38da879700 1 -- 172.21.15.143:0/767894832 >> v1:172.21.15.143:6789/0 conn(0x7f38d413fce0 legacy=0x7f38d4148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38da879700 1 -- 172.21.15.143:0/767894832 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40a40a0 con 0x7f38d40a3770 2023-09-29T23:31:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38da879700 1 --2- 172.21.15.143:0/767894832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d40a3770 0x7f38d40b71e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f38d40a4290 tx=0x7f38c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410fb714593b711d server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38d9076700 1 -- 172.21.15.143:0/767894832 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc0017f0 con 0x7f38d40a3770 2023-09-29T23:31:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38d9076700 1 -- 172.21.15.143:0/767894832 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8011420 con 0x7f38d40a3770 2023-09-29T23:31:27.856 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.850+0000 7f38d9076700 1 -- 172.21.15.143:0/767894832 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c80117c0 con 0x7f38d40a3770 2023-09-29T23:31:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 -- 172.21.15.143:0/767894832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d40a3770 msgr2=0x7f38d40b71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 --2- 172.21.15.143:0/767894832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d40a3770 0x7f38d40b71e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f38d40a4290 tx=0x7f38c80091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 -- 172.21.15.143:0/767894832 shutdown_connections 2023-09-29T23:31:27.857 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 --2- 172.21.15.143:0/767894832 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d40a3770 0x7f38d40b71e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 -- 172.21.15.143:0/767894832 >> 172.21.15.143:0/767894832 conn(0x7f38d4066c90 msgr2=0x7f38d4067090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 -- 172.21.15.143:0/767894832 shutdown_connections 2023-09-29T23:31:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 -- 172.21.15.143:0/767894832 wait complete. 2023-09-29T23:31:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 Processor -- start 2023-09-29T23:31:27.858 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 -- start start 2023-09-29T23:31:27.860 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d40a3770 0x7f38d409b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d40a5310 0x7f38d413c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f38d413fce0 0x7f38d413cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d413ee60 con 0x7f38d40a5310 2023-09-29T23:31:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d406d6d0 con 0x7f38d40a3770 2023-09-29T23:31:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38db07a700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d406c640 con 0x7f38d413fce0 2023-09-29T23:31:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38d9877700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d40a5310 0x7f38d413c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38d9877700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d40a5310 0x7f38d413c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54780/0 (socket says 172.21.15.143:54780) 2023-09-29T23:31:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38d9877700 1 -- 172.21.15.143:0/932115860 learned_addr learned my addr 172.21.15.143:0/932115860 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.854+0000 7f38da879700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f38d413fce0 0x7f38d413cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38da078700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d40a3770 0x7f38d409b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:27.861 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38d9877700 1 -- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f38d413fce0 msgr2=0x7f38d413cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:27.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38d9877700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f38d413fce0 0x7f38d413cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:27.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38d9877700 1 -- 172.21.15.143:0/932115860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d40a3770 msgr2=0x7f38d409b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:27.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38d9877700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d40a3770 0x7f38d409b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:27.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38d9877700 1 -- 172.21.15.143:0/932115860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c8009420 con 0x7f38d40a5310 2023-09-29T23:31:27.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38d9877700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d40a5310 0x7f38d413c980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00cbf0 tx=0x7f38cc007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:27.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38d27fc700 1 -- 172.21.15.143:0/932115860 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc004e90 con 0x7f38d40a5310 2023-09-29T23:31:27.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38d27fc700 1 -- 172.21.15.143:0/932115860 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38cc00a660 con 0x7f38d40a5310 2023-09-29T23:31:27.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38db07a700 1 -- 172.21.15.143:0/932115860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d413db10 con 0x7f38d40a5310 2023-09-29T23:31:27.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38d27fc700 1 -- 172.21.15.143:0/932115860 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00aa20 con 0x7f38d40a5310 2023-09-29T23:31:27.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38db07a700 1 -- 172.21.15.143:0/932115860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d409b840 con 0x7f38d40a5310 2023-09-29T23:31:27.863 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.858+0000 7f38db07a700 1 -- 172.21.15.143:0/932115860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d406c800 con 0x7f38d40a5310 2023-09-29T23:31:27.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.862+0000 7f38d27fc700 1 -- 172.21.15.143:0/932115860 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f38cc007150 con 0x7f38d40a5310 2023-09-29T23:31:27.866 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.862+0000 7f38d27fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:27.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.862+0000 7f38d27fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:27.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.862+0000 7f38d27fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:27.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.862+0000 7f38d27fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:27.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.862+0000 7f38d27fc700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f38c008e8b0 0x7f38c0090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:27.867 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.862+0000 7f38da078700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f38c008e8b0 0x7f38c0090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:27.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.862+0000 7f38d27fc700 1 -- 172.21.15.143:0/932115860 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f38cc0b41e0 con 0x7f38d40a5310 2023-09-29T23:31:27.868 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.862+0000 7f38da078700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f38c008e8b0 0x7f38c0090d60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f38c4007fd0 tx=0x7f38c4007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:27.874 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:27.870+0000 7f38d27fc700 1 -- 172.21.15.143:0/932115860 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f38cc084b90 con 0x7f38d40a5310 2023-09-29T23:31:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.062+0000 7f38db07a700 1 -- 172.21.15.143:0/932115860 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f38d405ecb0 con 0x7f38d40a5310 2023-09-29T23:31:28.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.062+0000 7f38d27fc700 1 -- 172.21.15.143:0/932115860 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f38cc0881b0 con 0x7f38d40a5310 2023-09-29T23:31:28.066 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:28.066 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.062+0000 7f38db07a700 10 mgrc shutdown 2023-09-29T23:31:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.062+0000 7f38db07a700 1 -- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f38c008e8b0 msgr2=0x7f38c0090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.062+0000 7f38db07a700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f38c008e8b0 0x7f38c0090d60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f38c4007fd0 tx=0x7f38c4007d50 comp rx=0 tx=0).stop 2023-09-29T23:31:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.062+0000 7f38db07a700 1 -- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d40a5310 msgr2=0x7f38d413c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.062+0000 7f38db07a700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d40a5310 0x7f38d413c980 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00cbf0 tx=0x7f38cc007da0 comp rx=0 tx=0).stop 2023-09-29T23:31:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.066+0000 7f38db07a700 1 -- 172.21.15.143:0/932115860 shutdown_connections 2023-09-29T23:31:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.066+0000 7f38db07a700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f38c008e8b0 0x7f38c0090d60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.066+0000 7f38db07a700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f38d413fce0 0x7f38d413cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.066+0000 7f38db07a700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f38d40a5310 0x7f38d413c980 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.066+0000 7f38db07a700 1 --2- 172.21.15.143:0/932115860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f38d40a3770 0x7f38d409b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:28.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.066+0000 7f38db07a700 1 -- 172.21.15.143:0/932115860 >> 172.21.15.143:0/932115860 conn(0x7f38d4066c90 msgr2=0x7f38d40a7660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:28.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.066+0000 7f38db07a700 1 -- 172.21.15.143:0/932115860 shutdown_connections 2023-09-29T23:31:28.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:28.066+0000 7f38db07a700 1 -- 172.21.15.143:0/932115860 wait complete. 2023-09-29T23:31:31.081 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:31.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a07fff700 1 Processor -- start 2023-09-29T23:31:31.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a07fff700 1 -- start start 2023-09-29T23:31:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a07fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a081363f0 0x7f3a081367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a07fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3a080a4f80 0x7f3a080a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a07fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a080a5ed0 con 0x7f3a080a6350 2023-09-29T23:31:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a07fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a080a6720 con 0x7f3a080a4f80 2023-09-29T23:31:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a07fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a080a5960 con 0x7f3a081363f0 2023-09-29T23:31:31.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a067fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a081363f0 0x7f3a081367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a067fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a081363f0 0x7f3a081367c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:45974/0 (socket says 172.21.15.143:45974) 2023-09-29T23:31:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a067fc700 1 -- 172.21.15.143:0/550440158 learned_addr learned my addr 172.21.15.143:0/550440158 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a077fe700 1 --2- 172.21.15.143:0/550440158 >> v2:172.21.15.156:3300/0 conn(0x7f3a080a4f80 0x7f3a080a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a067fc700 1 -- 172.21.15.143:0/550440158 >> v2:172.21.15.156:3300/0 conn(0x7f3a080a4f80 msgr2=0x7f3a080a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a067fc700 1 --2- 172.21.15.143:0/550440158 >> v2:172.21.15.156:3300/0 conn(0x7f3a080a4f80 0x7f3a080a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a067fc700 1 -- 172.21.15.143:0/550440158 >> v1:172.21.15.156:6789/0 conn(0x7f3a080a6350 legacy=0x7f3a0806b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a067fc700 1 -- 172.21.15.143:0/550440158 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a080a5c10 con 0x7f3a081363f0 2023-09-29T23:31:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a077fe700 1 --2- 172.21.15.143:0/550440158 >> v2:172.21.15.156:3300/0 conn(0x7f3a080a4f80 0x7f3a080a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-29T23:31:31.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a067fc700 1 --2- 172.21.15.143:0/550440158 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a081363f0 0x7f3a081367c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0099b0 tx=0x7f39fc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2a67f223731d9db server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:31.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a05ffb700 1 -- 172.21.15.143:0/550440158 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00b860 con 0x7f3a081363f0 2023-09-29T23:31:31.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a05ffb700 1 -- 172.21.15.143:0/550440158 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc00b9c0 con 0x7f3a081363f0 2023-09-29T23:31:31.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.185+0000 7f3a05ffb700 1 -- 172.21.15.143:0/550440158 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00bd60 con 0x7f3a081363f0 2023-09-29T23:31:31.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 -- 172.21.15.143:0/550440158 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a081363f0 msgr2=0x7f3a081367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 --2- 172.21.15.143:0/550440158 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a081363f0 0x7f3a081367c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0099b0 tx=0x7f39fc009250 comp rx=0 tx=0).stop 2023-09-29T23:31:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 -- 172.21.15.143:0/550440158 shutdown_connections 2023-09-29T23:31:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 --2- 172.21.15.143:0/550440158 >> v2:172.21.15.156:3300/0 conn(0x7f3a080a4f80 0x7f3a080a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 --2- 172.21.15.143:0/550440158 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a081363f0 0x7f3a081367c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 -- 172.21.15.143:0/550440158 >> 172.21.15.143:0/550440158 conn(0x7f3a080b6930 msgr2=0x7f3a080b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 -- 172.21.15.143:0/550440158 shutdown_connections 2023-09-29T23:31:31.194 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 -- 172.21.15.143:0/550440158 wait complete. 2023-09-29T23:31:31.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 Processor -- start 2023-09-29T23:31:31.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 -- start start 2023-09-29T23:31:31.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a080a4f80 0x7f3a081338f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:31.196 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a080a6350 0x7f3a08142940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a081363f0 0x7f3a08152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08132eb0 con 0x7f3a080a4f80 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a08143230 con 0x7f3a081363f0 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a07fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0814ffb0 con 0x7f3a080a6350 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a067fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a080a6350 0x7f3a08142940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a06ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a080a4f80 0x7f3a081338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a067fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a080a6350 0x7f3a08142940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:45986/0 (socket says 172.21.15.143:45986) 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a067fc700 1 -- 172.21.15.143:0/861233429 learned_addr learned my addr 172.21.15.143:0/861233429 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a077fe700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a081363f0 0x7f3a08152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a067fc700 1 -- 172.21.15.143:0/861233429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a081363f0 msgr2=0x7f3a08152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a067fc700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a081363f0 0x7f3a08152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:31.197 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a067fc700 1 -- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a080a4f80 msgr2=0x7f3a081338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:31.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a067fc700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a080a4f80 0x7f3a081338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:31.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a067fc700 1 -- 172.21.15.143:0/861233429 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc0094f0 con 0x7f3a080a6350 2023-09-29T23:31:31.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f3a067fc700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a080a6350 0x7f3a08142940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009290 tx=0x7f39fc017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:31.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.189+0000 7f39eeffd700 1 -- 172.21.15.143:0/861233429 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc011e40 con 0x7f3a080a6350 2023-09-29T23:31:31.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.193+0000 7f3a07fff700 1 -- 172.21.15.143:0/861233429 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a08150270 con 0x7f3a080a6350 2023-09-29T23:31:31.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.193+0000 7f39eeffd700 1 -- 172.21.15.143:0/861233429 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc024440 con 0x7f3a080a6350 2023-09-29T23:31:31.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.193+0000 7f39eeffd700 1 -- 172.21.15.143:0/861233429 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc024800 con 0x7f3a080a6350 2023-09-29T23:31:31.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.193+0000 7f3a07fff700 1 -- 172.21.15.143:0/861233429 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a081333b0 con 0x7f3a080a6350 2023-09-29T23:31:31.198 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.193+0000 7f3a07fff700 1 -- 172.21.15.143:0/861233429 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a08133610 con 0x7f3a080a6350 2023-09-29T23:31:31.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.197+0000 7f39eeffd700 1 -- 172.21.15.143:0/861233429 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f39fc01a420 con 0x7f3a080a6350 2023-09-29T23:31:31.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.197+0000 7f39eeffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:31.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.197+0000 7f39eeffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:31.201 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.197+0000 7f39eeffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.197+0000 7f39eeffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.197+0000 7f39eeffd700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39f408e830 0x7f39f4090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.197+0000 7f39eeffd700 1 -- 172.21.15.143:0/861233429 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f39fc0b6c20 con 0x7f3a080a6350 2023-09-29T23:31:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.197+0000 7f3a06ffd700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39f408e830 0x7f39f4090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:31.202 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.197+0000 7f3a06ffd700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39f408e830 0x7f39f4090ce0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f39f8007fd0 tx=0x7f39f80016f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:31.208 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.205+0000 7f39eeffd700 1 -- 172.21.15.143:0/861233429 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f39fc0875d0 con 0x7f3a080a6350 2023-09-29T23:31:31.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.401+0000 7f3a07fff700 1 -- 172.21.15.143:0/861233429 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3a08133610 con 0x7f3a080a6350 2023-09-29T23:31:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.401+0000 7f39eeffd700 1 -- 172.21.15.143:0/861233429 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f3a08133610 con 0x7f3a080a6350 2023-09-29T23:31:31.408 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:31.408 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:31.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 10 mgrc shutdown 2023-09-29T23:31:31.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 -- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39f408e830 msgr2=0x7f39f4090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39f408e830 0x7f39f4090ce0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f39f8007fd0 tx=0x7f39f80016f0 comp rx=0 tx=0).stop 2023-09-29T23:31:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 -- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a080a6350 msgr2=0x7f3a08142940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a080a6350 0x7f3a08142940 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009290 tx=0x7f39fc017000 comp rx=0 tx=0).stop 2023-09-29T23:31:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 -- 172.21.15.143:0/861233429 shutdown_connections 2023-09-29T23:31:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f39f408e830 0x7f39f4090ce0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a081363f0 0x7f3a08152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3a080a6350 0x7f3a08142940 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 --2- 172.21.15.143:0/861233429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3a080a4f80 0x7f3a081338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 -- 172.21.15.143:0/861233429 >> 172.21.15.143:0/861233429 conn(0x7f3a080b6930 msgr2=0x7f3a080a6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 -- 172.21.15.143:0/861233429 shutdown_connections 2023-09-29T23:31:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:31.405+0000 7f3a07fff700 1 -- 172.21.15.143:0/861233429 wait complete. 2023-09-29T23:31:34.424 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:34.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 Processor -- start 2023-09-29T23:31:34.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 -- start start 2023-09-29T23:31:34.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffa08062ed0 0x7ffa080a1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:34.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ffa080632a0 0x7ffa0809f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:34.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa0813cdf0 0x7ffa08140180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:34.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0813ca00 con 0x7ffa08062ed0 2023-09-29T23:31:34.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0813a4b0 con 0x7ffa080632a0 2023-09-29T23:31:34.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080a1760 con 0x7ffa0813cdf0 2023-09-29T23:31:34.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa06ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffa08062ed0 0x7ffa080a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:34.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa077fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa0813cdf0 0x7ffa08140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:34.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa06ffd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7ffa08062ed0 0x7ffa080a1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59474/0 (socket says 172.21.15.143:59474) 2023-09-29T23:31:34.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa06ffd700 1 -- 172.21.15.143:0/2951747661 learned_addr learned my addr 172.21.15.143:0/2951747661 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:34.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ff9fe7fc700 1 --2- 172.21.15.143:0/2951747661 >> v2:172.21.15.156:3300/0 conn(0x7ffa080632a0 0x7ffa0809f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:34.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa06ffd700 1 -- 172.21.15.143:0/2951747661 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa0813cdf0 msgr2=0x7ffa08140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:34.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa06ffd700 1 --2- 172.21.15.143:0/2951747661 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa0813cdf0 0x7ffa08140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:34.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa06ffd700 1 -- 172.21.15.143:0/2951747661 >> v2:172.21.15.156:3300/0 conn(0x7ffa080632a0 msgr2=0x7ffa0809f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:34.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa06ffd700 1 --2- 172.21.15.143:0/2951747661 >> v2:172.21.15.156:3300/0 conn(0x7ffa080632a0 0x7ffa0809f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:34.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa06ffd700 1 -- 172.21.15.143:0/2951747661 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0813d7c0 con 0x7ffa08062ed0 2023-09-29T23:31:34.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa06ffd700 1 --2- 172.21.15.143:0/2951747661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa08062ed0 0x7ffa080a1220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffa0813d9b0 tx=0x7ff9f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf9117c89f3286d server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa067fc700 1 -- 172.21.15.143:0/2951747661 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8009e70 con 0x7ffa08062ed0 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa067fc700 1 -- 172.21.15.143:0/2951747661 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f800b830 con 0x7ffa08062ed0 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa067fc700 1 -- 172.21.15.143:0/2951747661 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800bbf0 con 0x7ffa08062ed0 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 -- 172.21.15.143:0/2951747661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa08062ed0 msgr2=0x7ffa080a1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 --2- 172.21.15.143:0/2951747661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa08062ed0 0x7ffa080a1220 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffa0813d9b0 tx=0x7ff9f8009250 comp rx=0 tx=0).stop 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 -- 172.21.15.143:0/2951747661 shutdown_connections 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 --2- 172.21.15.143:0/2951747661 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa0813cdf0 0x7ffa08140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 --2- 172.21.15.143:0/2951747661 >> v2:172.21.15.156:3300/0 conn(0x7ffa080632a0 0x7ffa0809f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 --2- 172.21.15.143:0/2951747661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa08062ed0 0x7ffa080a1220 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 -- 172.21.15.143:0/2951747661 >> 172.21.15.143:0/2951747661 conn(0x7ffa0806ab30 msgr2=0x7ffa0806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 -- 172.21.15.143:0/2951747661 shutdown_connections 2023-09-29T23:31:34.541 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.533+0000 7ffa07fff700 1 -- 172.21.15.143:0/2951747661 wait complete. 2023-09-29T23:31:34.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa07fff700 1 Processor -- start 2023-09-29T23:31:34.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa07fff700 1 -- start start 2023-09-29T23:31:34.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa08062ed0 0x7ffa081415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:34.543 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa080632a0 0x7ffa080aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa07fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa0813cdf0 0x7ffa080aa610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa07fff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080ade10 con 0x7ffa080632a0 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa07fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080b43a0 con 0x7ffa08062ed0 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa07fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080aac50 con 0x7ffa0813cdf0 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9fe7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa080632a0 0x7ffa080aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa077fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa0813cdf0 0x7ffa080aa610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9fe7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa080632a0 0x7ffa080aa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59482/0 (socket says 172.21.15.143:59482) 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa077fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa0813cdf0 0x7ffa080aa610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:32792/0 (socket says 172.21.15.143:32792) 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9fe7fc700 1 -- 172.21.15.143:0/1969521834 learned_addr learned my addr 172.21.15.143:0/1969521834 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa06ffd700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa08062ed0 0x7ffa081415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9fe7fc700 1 -- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa0813cdf0 msgr2=0x7ffa080aa610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:34.544 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9fe7fc700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa0813cdf0 0x7ffa080aa610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:34.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9fe7fc700 1 -- 172.21.15.143:0/1969521834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa08062ed0 msgr2=0x7ffa081415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:34.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9fe7fc700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa08062ed0 0x7ffa081415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:34.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9fe7fc700 1 -- 172.21.15.143:0/1969521834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f80094f0 con 0x7ffa080632a0 2023-09-29T23:31:34.545 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9fe7fc700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa080632a0 0x7ffa080aa0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000d860 tx=0x7ff9f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:34.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9ff7fe700 1 -- 172.21.15.143:0/1969521834 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f00125f0 con 0x7ffa080632a0 2023-09-29T23:31:34.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9ff7fe700 1 -- 172.21.15.143:0/1969521834 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0012750 con 0x7ffa080632a0 2023-09-29T23:31:34.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa07fff700 1 -- 172.21.15.143:0/1969521834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa080aaf10 con 0x7ffa080632a0 2023-09-29T23:31:34.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ff9ff7fe700 1 -- 172.21.15.143:0/1969521834 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0012af0 con 0x7ffa080632a0 2023-09-29T23:31:34.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.537+0000 7ffa07fff700 1 -- 172.21.15.143:0/1969521834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa080a2640 con 0x7ffa080632a0 2023-09-29T23:31:34.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.541+0000 7ffa07fff700 1 -- 172.21.15.143:0/1969521834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa080adf80 con 0x7ffa080632a0 2023-09-29T23:31:34.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.541+0000 7ff9ff7fe700 1 -- 172.21.15.143:0/1969521834 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7ff9f0013480 con 0x7ffa080632a0 2023-09-29T23:31:34.547 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.541+0000 7ff9ff7fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:34.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.541+0000 7ff9ff7fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:34.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.541+0000 7ff9ff7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:34.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.541+0000 7ff9ff7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:34.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.541+0000 7ff9ff7fe700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff9e808e8b0 0x7ff9e8090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:34.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.541+0000 7ff9ff7fe700 1 -- 172.21.15.143:0/1969521834 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7ff9f00b4c50 con 0x7ffa080632a0 2023-09-29T23:31:34.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.541+0000 7ffa06ffd700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff9e808e8b0 0x7ff9e8090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:34.548 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.545+0000 7ffa06ffd700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff9e808e8b0 0x7ff9e8090d60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa0813d9b0 tx=0x7ff9f8016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:34.554 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.549+0000 7ff9ff7fe700 1 -- 172.21.15.143:0/1969521834 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff9f0085600 con 0x7ffa080632a0 2023-09-29T23:31:34.751 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.745+0000 7ffa07fff700 1 -- 172.21.15.143:0/1969521834 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ffa0805ecb0 con 0x7ffa080632a0 2023-09-29T23:31:34.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.745+0000 7ff9ff7fe700 1 -- 172.21.15.143:0/1969521834 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7ff9f0088c20 con 0x7ffa080632a0 2023-09-29T23:31:34.752 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:34.753 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:34.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 10 mgrc shutdown 2023-09-29T23:31:34.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 -- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff9e808e8b0 msgr2=0x7ff9e8090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:34.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff9e808e8b0 0x7ff9e8090d60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffa0813d9b0 tx=0x7ff9f8016000 comp rx=0 tx=0).stop 2023-09-29T23:31:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 -- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa080632a0 msgr2=0x7ffa080aa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa080632a0 0x7ffa080aa0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000d860 tx=0x7ff9f0008040 comp rx=0 tx=0).stop 2023-09-29T23:31:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 -- 172.21.15.143:0/1969521834 shutdown_connections 2023-09-29T23:31:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff9e808e8b0 0x7ff9e8090d60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ffa0813cdf0 0x7ffa080aa610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa080632a0 0x7ffa080aa0d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 --2- 172.21.15.143:0/1969521834 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa08062ed0 0x7ffa081415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 -- 172.21.15.143:0/1969521834 >> 172.21.15.143:0/1969521834 conn(0x7ffa0806ab30 msgr2=0x7ffa0813f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 -- 172.21.15.143:0/1969521834 shutdown_connections 2023-09-29T23:31:34.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:34.749+0000 7ffa07fff700 1 -- 172.21.15.143:0/1969521834 wait complete. 2023-09-29T23:31:37.771 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:37.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd36b3700 1 Processor -- start 2023-09-29T23:31:37.886 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd36b3700 1 -- start start 2023-09-29T23:31:37.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd36b3700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8bcc0a63f0 0x7f8bcc06c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:37.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd36b3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc0a67c0 con 0x7f8bcc06cbc0 2023-09-29T23:31:37.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd36b3700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc06c3c0 con 0x7f8bcc132bc0 2023-09-29T23:31:37.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd36b3700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bcc06cfb0 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd0c4e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8bcc0a63f0 0x7f8bcc06c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:37.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd0c4e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8bcc0a63f0 0x7f8bcc06c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59488/0 (socket says 172.21.15.143:59488) 2023-09-29T23:31:37.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd1c50700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f8bcc06cbc0 0x7f8bcc06b730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57732/0 (socket says 172.21.15.143:57732) 2023-09-29T23:31:37.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd0c4e700 1 -- 172.21.15.143:0/964427823 learned_addr learned my addr 172.21.15.143:0/964427823 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:37.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd0c4e700 1 -- 172.21.15.143:0/964427823 >> v1:172.21.15.156:6789/0 conn(0x7f8bcc132bc0 legacy=0x7f8bcc0b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:37.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd0c4e700 1 -- 172.21.15.143:0/964427823 >> v1:172.21.15.143:6789/0 conn(0x7f8bcc06cbc0 legacy=0x7f8bcc06b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:37.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd0c4e700 1 -- 172.21.15.143:0/964427823 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc06ad00 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bc3fff700 1 -- 172.21.15.143:0/964427823 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776404940 0 0) 0x7f8bcc0a67c0 con 0x7f8bcc06cbc0 2023-09-29T23:31:37.889 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd0c4e700 1 --2- 172.21.15.143:0/964427823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bcc0a63f0 0x7f8bcc06c680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800bc50 tx=0x7f8bc800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65518bf804340ddc server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:37.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bc3fff700 1 -- 172.21.15.143:0/964427823 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc800be60 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bc3fff700 1 -- 172.21.15.143:0/964427823 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc800de00 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bc3fff700 1 -- 172.21.15.143:0/964427823 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc80196b0 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd36b3700 1 -- 172.21.15.143:0/964427823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bcc0a63f0 msgr2=0x7f8bcc06c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:37.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.881+0000 7f8bd36b3700 1 --2- 172.21.15.143:0/964427823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bcc0a63f0 0x7f8bcc06c680 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800bc50 tx=0x7f8bc800b4f0 comp rx=0 tx=0).stop 2023-09-29T23:31:37.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 -- 172.21.15.143:0/964427823 shutdown_connections 2023-09-29T23:31:37.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 --2- 172.21.15.143:0/964427823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bcc0a63f0 0x7f8bcc06c680 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:37.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 -- 172.21.15.143:0/964427823 >> 172.21.15.143:0/964427823 conn(0x7f8bcc067dc0 msgr2=0x7f8bcc0b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:37.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 -- 172.21.15.143:0/964427823 shutdown_connections 2023-09-29T23:31:37.890 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 -- 172.21.15.143:0/964427823 wait complete. 2023-09-29T23:31:37.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 Processor -- start 2023-09-29T23:31:37.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 -- start start 2023-09-29T23:31:37.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bcc06cbc0 0x7f8bcc12e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:37.892 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bcc0a63f0 0x7f8bcc0a80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8bcc132bc0 0x7f8bcc13e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0a8600 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc12b2c0 con 0x7f8bcc06cbc0 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc13daf0 con 0x7f8bcc132bc0 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd0c4e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bcc0a63f0 0x7f8bcc0a80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd0c4e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bcc0a63f0 0x7f8bcc0a80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59496/0 (socket says 172.21.15.143:59496) 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd0c4e700 1 -- 172.21.15.143:0/2429494592 learned_addr learned my addr 172.21.15.143:0/2429494592 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd1c50700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8bcc132bc0 0x7f8bcc13e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd144f700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bcc06cbc0 0x7f8bcc12e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd0c4e700 1 -- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8bcc132bc0 msgr2=0x7f8bcc13e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd0c4e700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8bcc132bc0 0x7f8bcc13e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:37.893 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd0c4e700 1 -- 172.21.15.143:0/2429494592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bcc06cbc0 msgr2=0x7f8bcc12e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:37.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd0c4e700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bcc06cbc0 0x7f8bcc12e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:37.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd0c4e700 1 -- 172.21.15.143:0/2429494592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc800b790 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd1c50700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8bcc132bc0 0x7f8bcc13e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:31:37.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd0c4e700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bcc0a63f0 0x7f8bcc0a80c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc06e5a0 tx=0x7f8bc801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:37.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bc17fa700 1 -- 172.21.15.143:0/2429494592 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8010540 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bc17fa700 1 -- 172.21.15.143:0/2429494592 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc80106c0 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 -- 172.21.15.143:0/2429494592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc13dd10 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bc17fa700 1 -- 172.21.15.143:0/2429494592 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8010a80 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.885+0000 7f8bd36b3700 1 -- 172.21.15.143:0/2429494592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc13e9a0 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.889+0000 7f8bd36b3700 1 -- 172.21.15.143:0/2429494592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc05ecb0 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.889+0000 7f8bc17fa700 1 -- 172.21.15.143:0/2429494592 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f8bc8022030 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.889+0000 7f8bc17fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:37.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.889+0000 7f8bc17fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:37.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.889+0000 7f8bc17fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:37.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.889+0000 7f8bc17fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:37.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.889+0000 7f8bc17fa700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8bb008e8b0 0x7f8bb0090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:37.896 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.893+0000 7f8bd144f700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8bb008e8b0 0x7f8bb0090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:37.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.893+0000 7f8bc17fa700 1 -- 172.21.15.143:0/2429494592 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f8bc8014070 con 0x7f8bcc0a63f0 2023-09-29T23:31:37.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.893+0000 7f8bd144f700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8bb008e8b0 0x7f8bb0090d60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0037c0 tx=0x7f8bbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:37.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:37.897+0000 7f8bc17fa700 1 -- 172.21.15.143:0/2429494592 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8bc808a110 con 0x7f8bcc0a63f0 2023-09-29T23:31:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.093+0000 7f8bd36b3700 1 -- 172.21.15.143:0/2429494592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8bcc13f250 con 0x7f8bcc0a63f0 2023-09-29T23:31:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.093+0000 7f8bc17fa700 1 -- 172.21.15.143:0/2429494592 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f8bc808d730 con 0x7f8bcc0a63f0 2023-09-29T23:31:38.099 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:38.099 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:38.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.093+0000 7f8bd36b3700 10 mgrc shutdown 2023-09-29T23:31:38.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 -- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8bb008e8b0 msgr2=0x7f8bb0090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:38.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8bb008e8b0 0x7f8bb0090d60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0037c0 tx=0x7f8bbc009040 comp rx=0 tx=0).stop 2023-09-29T23:31:38.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 -- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bcc0a63f0 msgr2=0x7f8bcc0a80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:38.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bcc0a63f0 0x7f8bcc0a80c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc06e5a0 tx=0x7f8bc801b000 comp rx=0 tx=0).stop 2023-09-29T23:31:38.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 -- 172.21.15.143:0/2429494592 shutdown_connections 2023-09-29T23:31:38.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8bb008e8b0 0x7f8bb0090d60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:38.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8bcc132bc0 0x7f8bcc13e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:38.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bcc0a63f0 0x7f8bcc0a80c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:38.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 --2- 172.21.15.143:0/2429494592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bcc06cbc0 0x7f8bcc12e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:38.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 -- 172.21.15.143:0/2429494592 >> 172.21.15.143:0/2429494592 conn(0x7f8bcc067dc0 msgr2=0x7f8bcc12ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:38.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 -- 172.21.15.143:0/2429494592 shutdown_connections 2023-09-29T23:31:38.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:38.097+0000 7f8bd36b3700 1 -- 172.21.15.143:0/2429494592 wait complete. 2023-09-29T23:31:41.112 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:41.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 Processor -- start 2023-09-29T23:31:41.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 -- start start 2023-09-29T23:31:41.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f419c06f200 0x7f419c0a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:41.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c0b0590 0x7f419c09c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:41.229 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f419c0b09c0 con 0x7f419c06ee30 2023-09-29T23:31:41.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419c09cb90 con 0x7f419c06f200 2023-09-29T23:31:41.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c0a5330 con 0x7f419c0b0590 2023-09-29T23:31:41.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a1b65700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c0b0590 0x7f419c09c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:41.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a0b63700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f419c06f200 0x7f419c0a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:41.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a1b65700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c0b0590 0x7f419c09c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:32816/0 (socket says 172.21.15.143:32816) 2023-09-29T23:31:41.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a1b65700 1 -- 172.21.15.143:0/4042549516 learned_addr learned my addr 172.21.15.143:0/4042549516 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:41.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a0b63700 1 --2- 172.21.15.143:0/4042549516 >> v2:172.21.15.143:3300/0 conn(0x7f419c06f200 0x7f419c0a4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59500/0 (socket says 172.21.15.143:59500) 2023-09-29T23:31:41.230 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a0b63700 1 -- 172.21.15.143:0/4042549516 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c0b0590 msgr2=0x7f419c09c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:41.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a0b63700 1 --2- 172.21.15.143:0/4042549516 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c0b0590 0x7f419c09c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:41.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a0b63700 1 -- 172.21.15.143:0/4042549516 >> v1:172.21.15.156:6789/0 conn(0x7f419c06ee30 legacy=0x7f419c067ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:41.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a0b63700 1 -- 172.21.15.143:0/4042549516 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c0a55f0 con 0x7f419c06f200 2023-09-29T23:31:41.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a1b65700 1 --2- 172.21.15.143:0/4042549516 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c0b0590 0x7f419c09c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:41.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a0b63700 1 --2- 172.21.15.143:0/4042549516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f419c06f200 0x7f419c0a4ca0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4198009e60 tx=0x7f4198009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70379b1b6473a3cd server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:41.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f4193fff700 1 -- 172.21.15.143:0/4042549516 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419800e070 con 0x7f419c06f200 2023-09-29T23:31:41.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f4193fff700 1 -- 172.21.15.143:0/4042549516 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419800bb60 con 0x7f419c06f200 2023-09-29T23:31:41.231 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f4193fff700 1 -- 172.21.15.143:0/4042549516 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198012430 con 0x7f419c06f200 2023-09-29T23:31:41.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 -- 172.21.15.143:0/4042549516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f419c06f200 msgr2=0x7f419c0a4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:41.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 --2- 172.21.15.143:0/4042549516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f419c06f200 0x7f419c0a4ca0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4198009e60 tx=0x7f4198009780 comp rx=0 tx=0).stop 2023-09-29T23:31:41.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 -- 172.21.15.143:0/4042549516 shutdown_connections 2023-09-29T23:31:41.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 --2- 172.21.15.143:0/4042549516 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c0b0590 0x7f419c09c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:41.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 --2- 172.21.15.143:0/4042549516 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f419c06f200 0x7f419c0a4ca0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:41.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 -- 172.21.15.143:0/4042549516 >> 172.21.15.143:0/4042549516 conn(0x7f419c0b2ee0 msgr2=0x7f419c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:41.233 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 -- 172.21.15.143:0/4042549516 shutdown_connections 2023-09-29T23:31:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 -- 172.21.15.143:0/4042549516 wait complete. 2023-09-29T23:31:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 Processor -- start 2023-09-29T23:31:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.225+0000 7f41a35c8700 1 -- start start 2023-09-29T23:31:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a35c8700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c06ee30 0x7f419c0a2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a35c8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f419c06f200 0x7f419c0a0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a35c8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f419c0b0590 0x7f419c153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a35c8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c154c40 con 0x7f419c06f200 2023-09-29T23:31:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a35c8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c1457c0 con 0x7f419c0b0590 2023-09-29T23:31:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a35c8700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c0a2ec0 con 0x7f419c06ee30 2023-09-29T23:31:41.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a1364700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c06ee30 0x7f419c0a2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:41.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a0b63700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f419c06f200 0x7f419c0a0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:41.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a1364700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c06ee30 0x7f419c0a2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:32832/0 (socket says 172.21.15.143:32832) 2023-09-29T23:31:41.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a1364700 1 -- 172.21.15.143:0/413338628 learned_addr learned my addr 172.21.15.143:0/413338628 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:41.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a1b65700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f419c0b0590 0x7f419c153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:41.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a0b63700 1 -- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c06ee30 msgr2=0x7f419c0a2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:41.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a0b63700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c06ee30 0x7f419c0a2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:41.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a0b63700 1 -- 172.21.15.143:0/413338628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f419c0b0590 msgr2=0x7f419c153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:41.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a0b63700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f419c0b0590 0x7f419c153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:41.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a0b63700 1 -- 172.21.15.143:0/413338628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c0094f0 con 0x7f419c06f200 2023-09-29T23:31:41.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a1364700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c06ee30 0x7f419c0a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:41.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a0b63700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f419c06f200 0x7f419c0a0e90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4198016000 tx=0x7f4198019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:41.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41917fa700 1 -- 172.21.15.143:0/413338628 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198012d00 con 0x7f419c06f200 2023-09-29T23:31:41.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41917fa700 1 -- 172.21.15.143:0/413338628 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4198012e80 con 0x7f419c06f200 2023-09-29T23:31:41.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a35c8700 1 -- 172.21.15.143:0/413338628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4198009a20 con 0x7f419c06f200 2023-09-29T23:31:41.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41917fa700 1 -- 172.21.15.143:0/413338628 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41980276c0 con 0x7f419c06f200 2023-09-29T23:31:41.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a35c8700 1 -- 172.21.15.143:0/413338628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419c155020 con 0x7f419c06f200 2023-09-29T23:31:41.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a35c8700 1 -- 172.21.15.143:0/413338628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419c05ecb0 con 0x7f419c06f200 2023-09-29T23:31:41.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41917fa700 1 -- 172.21.15.143:0/413338628 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f4198024070 con 0x7f419c06f200 2023-09-29T23:31:41.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41917fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:41.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41917fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:41.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41917fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:41.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41917fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:41.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41917fa700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f418808e8b0 0x7f4188090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:41.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a1364700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f418808e8b0 0x7f4188090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:41.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41917fa700 1 -- 172.21.15.143:0/413338628 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f41980b6f30 con 0x7f419c06f200 2023-09-29T23:31:41.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.233+0000 7f41a1364700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f418808e8b0 0x7f4188090d60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4194001e00 tx=0x7f4194009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:41.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.241+0000 7f41917fa700 1 -- 172.21.15.143:0/413338628 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4198087980 con 0x7f419c06f200 2023-09-29T23:31:41.440 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.437+0000 7f41a35c8700 1 -- 172.21.15.143:0/413338628 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f419c09c0f0 con 0x7f419c06f200 2023-09-29T23:31:41.441 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41917fa700 1 -- 172.21.15.143:0/413338628 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f419808afa0 con 0x7f419c06f200 2023-09-29T23:31:41.441 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:41.441 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:41.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 10 mgrc shutdown 2023-09-29T23:31:41.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 -- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f418808e8b0 msgr2=0x7f4188090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:41.443 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f418808e8b0 0x7f4188090d60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4194001e00 tx=0x7f4194009040 comp rx=0 tx=0).stop 2023-09-29T23:31:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 -- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f419c06f200 msgr2=0x7f419c0a0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f419c06f200 0x7f419c0a0e90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4198016000 tx=0x7f4198019000 comp rx=0 tx=0).stop 2023-09-29T23:31:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 -- 172.21.15.143:0/413338628 shutdown_connections 2023-09-29T23:31:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f418808e8b0 0x7f4188090d60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f419c0b0590 0x7f419c153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f419c06f200 0x7f419c0a0e90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 --2- 172.21.15.143:0/413338628 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f419c06ee30 0x7f419c0a2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 -- 172.21.15.143:0/413338628 >> 172.21.15.143:0/413338628 conn(0x7f419c0b2ee0 msgr2=0x7f419c146a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 -- 172.21.15.143:0/413338628 shutdown_connections 2023-09-29T23:31:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:41.441+0000 7f41a35c8700 1 -- 172.21.15.143:0/413338628 wait complete. 2023-09-29T23:31:44.457 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:44.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81ed5cb700 1 Processor -- start 2023-09-29T23:31:44.571 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81ed5cb700 1 -- start start 2023-09-29T23:31:44.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81ed5cb700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 0x7f81e813d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:44.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81ed5cb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f81e813d990 0x7f81e8148700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:44.572 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81ed5cb700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e806e1d0 con 0x7f81e806de00 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81ed5cb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e809e560 con 0x7f81e813d990 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81ed5cb700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e80a6470 con 0x7f81e80a6910 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81e7fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f81e806de00 0x7f81e809fed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45482/0 (socket says 172.21.15.143:45482) 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81e7fff700 1 -- 172.21.15.143:0/4087786009 learned_addr learned my addr 172.21.15.143:0/4087786009 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81e77fe700 1 --2- 172.21.15.143:0/4087786009 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 0x7f81e813d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81ecdca700 1 --2- 172.21.15.143:0/4087786009 >> v2:172.21.15.156:3300/0 conn(0x7f81e813d990 0x7f81e8148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81e6ffd700 1 -- 172.21.15.143:0/4087786009 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 746737320 0 0) 0x7f81e806e1d0 con 0x7f81e806de00 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.569+0000 7f81e6ffd700 1 -- 172.21.15.143:0/4087786009 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d0003570 con 0x7f81e806de00 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81e77fe700 1 -- 172.21.15.143:0/4087786009 >> v2:172.21.15.156:3300/0 conn(0x7f81e813d990 msgr2=0x7f81e8148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81e77fe700 1 --2- 172.21.15.143:0/4087786009 >> v2:172.21.15.156:3300/0 conn(0x7f81e813d990 0x7f81e8148700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81e77fe700 1 -- 172.21.15.143:0/4087786009 >> v1:172.21.15.143:6789/0 conn(0x7f81e806de00 legacy=0x7f81e809fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:44.573 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81e6ffd700 1 -- 172.21.15.143:0/4087786009 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3287534472 0 0) 0x7f81d0003570 con 0x7f81e806de00 2023-09-29T23:31:44.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81e77fe700 1 -- 172.21.15.143:0/4087786009 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e809e800 con 0x7f81e80a6910 2023-09-29T23:31:44.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81ecdca700 1 --2- 172.21.15.143:0/4087786009 >> v2:172.21.15.156:3300/0 conn(0x7f81e813d990 0x7f81e8148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81e77fe700 1 --2- 172.21.15.143:0/4087786009 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 0x7f81e813d450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81e809e9f0 tx=0x7f81dc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7318dd37fe65587e server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81e6ffd700 1 -- 172.21.15.143:0/4087786009 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc011630 con 0x7f81e80a6910 2023-09-29T23:31:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81e6ffd700 1 -- 172.21.15.143:0/4087786009 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc011790 con 0x7f81e80a6910 2023-09-29T23:31:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81e6ffd700 1 -- 172.21.15.143:0/4087786009 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc011b50 con 0x7f81e80a6910 2023-09-29T23:31:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81ed5cb700 1 -- 172.21.15.143:0/4087786009 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 msgr2=0x7f81e813d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81ed5cb700 1 --2- 172.21.15.143:0/4087786009 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 0x7f81e813d450 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81e809e9f0 tx=0x7f81dc0091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:44.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81ed5cb700 1 -- 172.21.15.143:0/4087786009 shutdown_connections 2023-09-29T23:31:44.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81ed5cb700 1 --2- 172.21.15.143:0/4087786009 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 0x7f81e813d450 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:44.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81ed5cb700 1 --2- 172.21.15.143:0/4087786009 >> v2:172.21.15.156:3300/0 conn(0x7f81e813d990 0x7f81e8148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:44.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81ed5cb700 1 -- 172.21.15.143:0/4087786009 >> 172.21.15.143:0/4087786009 conn(0x7f81e80b4640 msgr2=0x7f81e80b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:44.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81ed5cb700 1 -- 172.21.15.143:0/4087786009 shutdown_connections 2023-09-29T23:31:44.577 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81ed5cb700 1 -- 172.21.15.143:0/4087786009 wait complete. 2023-09-29T23:31:44.579 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81ed5cb700 1 Processor -- start 2023-09-29T23:31:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.573+0000 7f81ed5cb700 1 -- start start 2023-09-29T23:31:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ed5cb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e806de00 0x7f81e8141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ed5cb700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 0x7f81e8096b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ed5cb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e813d990 0x7f81e8151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ed5cb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8151c10 con 0x7f81e813d990 2023-09-29T23:31:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ed5cb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8151ed0 con 0x7f81e806de00 2023-09-29T23:31:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ed5cb700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8097040 con 0x7f81e80a6910 2023-09-29T23:31:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81e77fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 0x7f81e8096b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81e77fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 0x7f81e8096b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:55558/0 (socket says 172.21.15.143:55558) 2023-09-29T23:31:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81e77fe700 1 -- 172.21.15.143:0/130382677 learned_addr learned my addr 172.21.15.143:0/130382677 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:44.580 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ecdca700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e813d990 0x7f81e8151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81e7fff700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e806de00 0x7f81e8141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ecdca700 1 -- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 msgr2=0x7f81e8096b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ecdca700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 0x7f81e8096b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ecdca700 1 -- 172.21.15.143:0/130382677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e806de00 msgr2=0x7f81e8141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ecdca700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e806de00 0x7f81e8141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ecdca700 1 -- 172.21.15.143:0/130382677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d8009420 con 0x7f81e813d990 2023-09-29T23:31:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ecdca700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e813d990 0x7f81e8151090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f81d800e970 tx=0x7f81d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81cffff700 1 -- 172.21.15.143:0/130382677 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8013680 con 0x7f81e813d990 2023-09-29T23:31:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81cffff700 1 -- 172.21.15.143:0/130382677 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d80137e0 con 0x7f81e813d990 2023-09-29T23:31:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ed5cb700 1 -- 172.21.15.143:0/130382677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81dc009420 con 0x7f81e813d990 2023-09-29T23:31:44.581 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81cffff700 1 -- 172.21.15.143:0/130382677 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d8013b60 con 0x7f81e813d990 2023-09-29T23:31:44.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.577+0000 7f81ed5cb700 1 -- 172.21.15.143:0/130382677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e8142200 con 0x7f81e813d990 2023-09-29T23:31:44.582 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.581+0000 7f81ed5cb700 1 -- 172.21.15.143:0/130382677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e805ecb0 con 0x7f81e813d990 2023-09-29T23:31:44.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.581+0000 7f81cffff700 1 -- 172.21.15.143:0/130382677 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f81d8024430 con 0x7f81e813d990 2023-09-29T23:31:44.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.581+0000 7f81cffff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:44.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.585+0000 7f81cffff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:44.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.585+0000 7f81cffff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:44.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.585+0000 7f81cffff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:44.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.585+0000 7f81cffff700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f81d408e8e0 0x7f81d4090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:44.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.585+0000 7f81e7fff700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f81d408e8e0 0x7f81d4090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:44.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.585+0000 7f81cffff700 1 -- 172.21.15.143:0/130382677 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f81d80b4de0 con 0x7f81e813d990 2023-09-29T23:31:44.585 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.585+0000 7f81e7fff700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f81d408e8e0 0x7f81d4090d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f81e0007fd0 tx=0x7f81e0007e70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:44.587 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.585+0000 7f81cffff700 1 -- 172.21.15.143:0/130382677 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f81d8085830 con 0x7f81e813d990 2023-09-29T23:31:44.779 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.777+0000 7f81ed5cb700 1 -- 172.21.15.143:0/130382677 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f81e8097260 con 0x7f81e813d990 2023-09-29T23:31:44.780 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.777+0000 7f81cffff700 1 -- 172.21.15.143:0/130382677 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f81d8088e50 con 0x7f81e813d990 2023-09-29T23:31:44.780 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:44.780 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:44.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 10 mgrc shutdown 2023-09-29T23:31:44.782 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 -- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f81d408e8e0 msgr2=0x7f81d4090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f81d408e8e0 0x7f81d4090d90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f81e0007fd0 tx=0x7f81e0007e70 comp rx=0 tx=0).stop 2023-09-29T23:31:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 -- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e813d990 msgr2=0x7f81e8151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e813d990 0x7f81e8151090 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f81d800e970 tx=0x7f81d80091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 -- 172.21.15.143:0/130382677 shutdown_connections 2023-09-29T23:31:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f81d408e8e0 0x7f81d4090d90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e813d990 0x7f81e8151090 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:44.783 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f81e80a6910 0x7f81e8096b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:44.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 --2- 172.21.15.143:0/130382677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81e806de00 0x7f81e8141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:44.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 -- 172.21.15.143:0/130382677 >> 172.21.15.143:0/130382677 conn(0x7f81e80b4640 msgr2=0x7f81e806fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:44.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 -- 172.21.15.143:0/130382677 shutdown_connections 2023-09-29T23:31:44.784 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:44.781+0000 7f81ed5cb700 1 -- 172.21.15.143:0/130382677 wait complete. 2023-09-29T23:31:47.800 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:47.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f475cfe2700 1 Processor -- start 2023-09-29T23:31:47.914 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f475cfe2700 1 -- start start 2023-09-29T23:31:47.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f475cfe2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 0x7f47580aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:47.915 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f475cfe2700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f47580a1470 0x7f475813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f475cfe2700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47580a19f0 con 0x7f475813a300 2023-09-29T23:31:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f475cfe2700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4758067290 con 0x7f47580a1470 2023-09-29T23:31:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f475cfe2700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f475809f560 con 0x7f4758066ec0 2023-09-29T23:31:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f4756ffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f475813a300 0x7f47580a0d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45494/0 (socket says 172.21.15.143:45494) 2023-09-29T23:31:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f4756ffd700 1 -- 172.21.15.143:0/2419174622 learned_addr learned my addr 172.21.15.143:0/2419174622 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:31:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f47577fe700 1 --2- 172.21.15.143:0/2419174622 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 0x7f47580aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f4757fff700 1 --2- 172.21.15.143:0/2419174622 >> v2:172.21.15.156:3300/0 conn(0x7f47580a1470 0x7f475813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f47567fc700 1 -- 172.21.15.143:0/2419174622 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151647483 0 0) 0x7f47580a19f0 con 0x7f475813a300 2023-09-29T23:31:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f47567fc700 1 -- 172.21.15.143:0/2419174622 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4740003570 con 0x7f475813a300 2023-09-29T23:31:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f4757fff700 1 -- 172.21.15.143:0/2419174622 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 msgr2=0x7f47580aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:47.916 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f4757fff700 1 --2- 172.21.15.143:0/2419174622 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 0x7f47580aee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f4757fff700 1 -- 172.21.15.143:0/2419174622 >> v1:172.21.15.143:6789/0 conn(0x7f475813a300 legacy=0x7f47580a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f4757fff700 1 -- 172.21.15.143:0/2419174622 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47580a20b0 con 0x7f47580a1470 2023-09-29T23:31:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f47577fe700 1 --2- 172.21.15.143:0/2419174622 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 0x7f47580aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:47.917 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f4757fff700 1 --2- 172.21.15.143:0/2419174622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47580a1470 0x7f475813a730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f47500098e0 tx=0x7f47500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44469d1dded0cfa8 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:47.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f47567fc700 1 -- 172.21.15.143:0/2419174622 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4750011630 con 0x7f47580a1470 2023-09-29T23:31:47.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.913+0000 7f47567fc700 1 -- 172.21.15.143:0/2419174622 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4750011790 con 0x7f47580a1470 2023-09-29T23:31:47.918 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.917+0000 7f47567fc700 1 -- 172.21.15.143:0/2419174622 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4750011b10 con 0x7f47580a1470 2023-09-29T23:31:47.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.917+0000 7f475cfe2700 1 -- 172.21.15.143:0/2419174622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47580a1470 msgr2=0x7f475813a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:47.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.917+0000 7f475cfe2700 1 --2- 172.21.15.143:0/2419174622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47580a1470 0x7f475813a730 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f47500098e0 tx=0x7f47500091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:47.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.917+0000 7f475cfe2700 1 -- 172.21.15.143:0/2419174622 shutdown_connections 2023-09-29T23:31:47.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.917+0000 7f475cfe2700 1 --2- 172.21.15.143:0/2419174622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47580a1470 0x7f475813a730 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:47.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.917+0000 7f475cfe2700 1 --2- 172.21.15.143:0/2419174622 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 0x7f47580aee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:47.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.917+0000 7f475cfe2700 1 -- 172.21.15.143:0/2419174622 >> 172.21.15.143:0/2419174622 conn(0x7f475806f200 msgr2=0x7f4758069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:47.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.917+0000 7f475cfe2700 1 -- 172.21.15.143:0/2419174622 shutdown_connections 2023-09-29T23:31:47.919 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.917+0000 7f475cfe2700 1 -- 172.21.15.143:0/2419174622 wait complete. 2023-09-29T23:31:47.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.917+0000 7f475cfe2700 1 Processor -- start 2023-09-29T23:31:47.920 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.917+0000 7f475cfe2700 1 -- start start 2023-09-29T23:31:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f475cfe2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 0x7f475806dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f475cfe2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47580a1470 0x7f4758137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f475cfe2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475813a300 0x7f4758138500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f475cfe2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4758137b50 con 0x7f47580a1470 2023-09-29T23:31:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f475cfe2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47581465c0 con 0x7f475813a300 2023-09-29T23:31:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f475cfe2700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4758146880 con 0x7f4758066ec0 2023-09-29T23:31:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f4756ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47580a1470 0x7f4758137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f4756ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47580a1470 0x7f4758137fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48432/0 (socket says 172.21.15.143:48432) 2023-09-29T23:31:47.922 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f47577fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 0x7f475806dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:47.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f4756ffd700 1 -- 172.21.15.143:0/887169438 learned_addr learned my addr 172.21.15.143:0/887169438 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:47.923 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f4757fff700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475813a300 0x7f4758138500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:47.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f4756ffd700 1 -- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 msgr2=0x7f475806dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:47.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f4756ffd700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 0x7f475806dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:47.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f4756ffd700 1 -- 172.21.15.143:0/887169438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475813a300 msgr2=0x7f4758138500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:47.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f4756ffd700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475813a300 0x7f4758138500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:47.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f4756ffd700 1 -- 172.21.15.143:0/887169438 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c009420 con 0x7f47580a1470 2023-09-29T23:31:47.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f4757fff700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475813a300 0x7f4758138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:47.924 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f47577fe700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 0x7f475806dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:47.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f4756ffd700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47580a1470 0x7f4758137fc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f474800cd00 tx=0x7f4748001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:47.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f473f7fe700 1 -- 172.21.15.143:0/887169438 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474800ce60 con 0x7f47580a1470 2023-09-29T23:31:47.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f475cfe2700 1 -- 172.21.15.143:0/887169438 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4750009420 con 0x7f47580a1470 2023-09-29T23:31:47.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f473f7fe700 1 -- 172.21.15.143:0/887169438 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474800aa60 con 0x7f47580a1470 2023-09-29T23:31:47.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f473f7fe700 1 -- 172.21.15.143:0/887169438 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474800ae00 con 0x7f47580a1470 2023-09-29T23:31:47.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.921+0000 7f475cfe2700 1 -- 172.21.15.143:0/887169438 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4758136da0 con 0x7f47580a1470 2023-09-29T23:31:47.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.925+0000 7f475cfe2700 1 -- 172.21.15.143:0/887169438 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475805ecb0 con 0x7f47580a1470 2023-09-29T23:31:47.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.925+0000 7f473f7fe700 1 -- 172.21.15.143:0/887169438 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f4748007300 con 0x7f47580a1470 2023-09-29T23:31:47.926 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.925+0000 7f473f7fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:47.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.925+0000 7f473f7fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:47.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.925+0000 7f473f7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:47.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.925+0000 7f473f7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:47.927 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.925+0000 7f473f7fe700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f474408e830 0x7f4744090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:47.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.925+0000 7f47577fe700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f474408e830 0x7f4744090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:47.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.925+0000 7f473f7fe700 1 -- 172.21.15.143:0/887169438 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f47480b4c80 con 0x7f47580a1470 2023-09-29T23:31:47.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.925+0000 7f47577fe700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f474408e830 0x7f4744090ce0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f474c011560 tx=0x7f474c0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:47.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:47.937+0000 7f473f7fe700 1 -- 172.21.15.143:0/887169438 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4748085630 con 0x7f47580a1470 2023-09-29T23:31:48.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.125+0000 7f475cfe2700 1 -- 172.21.15.143:0/887169438 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4758136fc0 con 0x7f47580a1470 2023-09-29T23:31:48.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f473f7fe700 1 -- 172.21.15.143:0/887169438 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f4748088c50 con 0x7f47580a1470 2023-09-29T23:31:48.129 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:48.129 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 10 mgrc shutdown 2023-09-29T23:31:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 -- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f474408e830 msgr2=0x7f4744090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f474408e830 0x7f4744090ce0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f474c011560 tx=0x7f474c0091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 -- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47580a1470 msgr2=0x7f4758137fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47580a1470 0x7f4758137fc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f474800cd00 tx=0x7f4748001b00 comp rx=0 tx=0).stop 2023-09-29T23:31:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 -- 172.21.15.143:0/887169438 shutdown_connections 2023-09-29T23:31:48.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f474408e830 0x7f4744090ce0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:48.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f475813a300 0x7f4758138500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:48.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f47580a1470 0x7f4758137fc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:48.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 --2- 172.21.15.143:0/887169438 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4758066ec0 0x7f475806dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:48.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 -- 172.21.15.143:0/887169438 >> 172.21.15.143:0/887169438 conn(0x7f475806f200 msgr2=0x7f47580b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:48.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 -- 172.21.15.143:0/887169438 shutdown_connections 2023-09-29T23:31:48.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:48.129+0000 7f475cfe2700 1 -- 172.21.15.143:0/887169438 wait complete. 2023-09-29T23:31:51.148 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:51.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.261+0000 7fd4ba315700 1 Processor -- start 2023-09-29T23:31:51.263 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.261+0000 7fd4ba315700 1 -- start start 2023-09-29T23:31:51.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd4b409f820 0x7fd4b41446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:51.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4b4068070 0x7fd4b4146c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:51.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4b40a4d50 con 0x7fd4b4067ca0 2023-09-29T23:31:51.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b409c320 con 0x7fd4b4068070 2023-09-29T23:31:51.266 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4b409c5c0 con 0x7fd4b409f820 2023-09-29T23:31:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4b88b2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4b4068070 0x7fd4b4146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4b88b2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd4b4068070 0x7fd4b4146c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48440/0 (socket says 172.21.15.143:48440) 2023-09-29T23:31:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4b88b2700 1 -- 172.21.15.143:0/1019627180 learned_addr learned my addr 172.21.15.143:0/1019627180 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ab7fe700 1 --2- 172.21.15.143:0/1019627180 >> v2:172.21.15.156:3300/0 conn(0x7fd4b409f820 0x7fd4b41446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4b88b2700 1 -- 172.21.15.143:0/1019627180 >> v2:172.21.15.156:3300/0 conn(0x7fd4b409f820 msgr2=0x7fd4b41446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4b88b2700 1 --2- 172.21.15.143:0/1019627180 >> v2:172.21.15.156:3300/0 conn(0x7fd4b409f820 0x7fd4b41446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4b88b2700 1 -- 172.21.15.143:0/1019627180 >> v1:172.21.15.156:6789/0 conn(0x7fd4b4067ca0 legacy=0x7fd4b409e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4b88b2700 1 -- 172.21.15.143:0/1019627180 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4b409eb40 con 0x7fd4b4068070 2023-09-29T23:31:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4b88b2700 1 --2- 172.21.15.143:0/1019627180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4b4068070 0x7fd4b4146c20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd4b406b130 tx=0x7fd4ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba8479f5b4c1319 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:51.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4b37fe700 1 -- 172.21.15.143:0/1019627180 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac00e070 con 0x7fd4b4068070 2023-09-29T23:31:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4b37fe700 1 -- 172.21.15.143:0/1019627180 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ac00ba50 con 0x7fd4b4068070 2023-09-29T23:31:51.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4b37fe700 1 -- 172.21.15.143:0/1019627180 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ac0092e0 con 0x7fd4b4068070 2023-09-29T23:31:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 -- 172.21.15.143:0/1019627180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4b4068070 msgr2=0x7fd4b4146c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 --2- 172.21.15.143:0/1019627180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4b4068070 0x7fd4b4146c20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd4b406b130 tx=0x7fd4ac009840 comp rx=0 tx=0).stop 2023-09-29T23:31:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 -- 172.21.15.143:0/1019627180 shutdown_connections 2023-09-29T23:31:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 --2- 172.21.15.143:0/1019627180 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4b4068070 0x7fd4b4146c20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 --2- 172.21.15.143:0/1019627180 >> v2:172.21.15.156:3300/0 conn(0x7fd4b409f820 0x7fd4b41446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 -- 172.21.15.143:0/1019627180 >> 172.21.15.143:0/1019627180 conn(0x7fd4b40b2ee0 msgr2=0x7fd4b40b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 -- 172.21.15.143:0/1019627180 shutdown_connections 2023-09-29T23:31:51.269 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.265+0000 7fd4ba315700 1 -- 172.21.15.143:0/1019627180 wait complete. 2023-09-29T23:31:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ba315700 1 Processor -- start 2023-09-29T23:31:51.270 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ba315700 1 -- start start 2023-09-29T23:31:51.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ba315700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4b4067ca0 0x7fd4b40b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:51.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ba315700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4b4068070 0x7fd4b40a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:51.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ba315700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b409f820 0x7fd4b413ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:51.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ba315700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b4140ff0 con 0x7fd4b4068070 2023-09-29T23:31:51.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ba315700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b40a1790 con 0x7fd4b409f820 2023-09-29T23:31:51.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ba315700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4b406b690 con 0x7fd4b4067ca0 2023-09-29T23:31:51.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4b4068070 0x7fd4b40a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:51.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4b3fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4b4067ca0 0x7fd4b40b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:51.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ab7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4b4068070 0x7fd4b40a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:48446/0 (socket says 172.21.15.143:48446) 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ab7fe700 1 -- 172.21.15.143:0/1135231382 learned_addr learned my addr 172.21.15.143:0/1135231382 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4b88b2700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b409f820 0x7fd4b413ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ab7fe700 1 -- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4b4067ca0 msgr2=0x7fd4b40b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ab7fe700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4b4067ca0 0x7fd4b40b14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ab7fe700 1 -- 172.21.15.143:0/1135231382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b409f820 msgr2=0x7fd4b413ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ab7fe700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b409f820 0x7fd4b413ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ab7fe700 1 -- 172.21.15.143:0/1135231382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac009ae0 con 0x7fd4b4068070 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4b3fff700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4b4067ca0 0x7fd4b40b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ab7fe700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4b4068070 0x7fd4b40a0d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00d960 tx=0x7fd49c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4b0ff9700 1 -- 172.21.15.143:0/1135231382 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c011670 con 0x7fd4b4068070 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ba315700 1 -- 172.21.15.143:0/1135231382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4b40a0870 con 0x7fd4b4068070 2023-09-29T23:31:51.273 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4b0ff9700 1 -- 172.21.15.143:0/1135231382 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49c0117f0 con 0x7fd4b4068070 2023-09-29T23:31:51.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4b0ff9700 1 -- 172.21.15.143:0/1135231382 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49c011bb0 con 0x7fd4b4068070 2023-09-29T23:31:51.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.269+0000 7fd4ba315700 1 -- 172.21.15.143:0/1135231382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4b40b1ce0 con 0x7fd4b4068070 2023-09-29T23:31:51.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.273+0000 7fd4ba315700 1 -- 172.21.15.143:0/1135231382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4b405ecb0 con 0x7fd4b4068070 2023-09-29T23:31:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.273+0000 7fd4b0ff9700 1 -- 172.21.15.143:0/1135231382 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fd49c023440 con 0x7fd4b4068070 2023-09-29T23:31:51.276 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.273+0000 7fd4b0ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.277+0000 7fd4b0ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.277+0000 7fd4b0ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.277+0000 7fd4b0ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:51.277 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.277+0000 7fd4b0ff9700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd49408e8b0 0x7fd494090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.277+0000 7fd4b3fff700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd49408e8b0 0x7fd494090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.277+0000 7fd4b0ff9700 1 -- 172.21.15.143:0/1135231382 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fd49c0b4b90 con 0x7fd4b4068070 2023-09-29T23:31:51.278 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.277+0000 7fd4b3fff700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd49408e8b0 0x7fd494090d60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4001d20 tx=0x7fd4a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:51.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.285+0000 7fd4b0ff9700 1 -- 172.21.15.143:0/1135231382 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd49c085550 con 0x7fd4b4068070 2023-09-29T23:31:51.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.481+0000 7fd4ba315700 1 -- 172.21.15.143:0/1135231382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd4b409c860 con 0x7fd4b4068070 2023-09-29T23:31:51.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.481+0000 7fd4b0ff9700 1 -- 172.21.15.143:0/1135231382 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fd49c01e080 con 0x7fd4b4068070 2023-09-29T23:31:51.482 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:51.483 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:51.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 10 mgrc shutdown 2023-09-29T23:31:51.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 -- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd49408e8b0 msgr2=0x7fd494090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:51.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd49408e8b0 0x7fd494090d60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd4a4001d20 tx=0x7fd4a4009040 comp rx=0 tx=0).stop 2023-09-29T23:31:51.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 -- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4b4068070 msgr2=0x7fd4b40a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:51.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4b4068070 0x7fd4b40a0d90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd49c00d960 tx=0x7fd49c008040 comp rx=0 tx=0).stop 2023-09-29T23:31:51.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 -- 172.21.15.143:0/1135231382 shutdown_connections 2023-09-29T23:31:51.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd49408e8b0 0x7fd494090d60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:51.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4b409f820 0x7fd4b413ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:51.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd4b4068070 0x7fd4b40a0d90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:51.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 --2- 172.21.15.143:0/1135231382 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4b4067ca0 0x7fd4b40b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:51.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 -- 172.21.15.143:0/1135231382 >> 172.21.15.143:0/1135231382 conn(0x7fd4b40b2ee0 msgr2=0x7fd4b406ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:51.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 -- 172.21.15.143:0/1135231382 shutdown_connections 2023-09-29T23:31:51.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:51.485+0000 7fd4ba315700 1 -- 172.21.15.143:0/1135231382 wait complete. 2023-09-29T23:31:54.496 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:54.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.605+0000 7fa59b1d9700 1 Processor -- start 2023-09-29T23:31:54.608 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 -- start start 2023-09-29T23:31:54.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fa594066ec0 0x7fa59409fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:54.609 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa594067290 con 0x7fa59406b680 2023-09-29T23:31:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5940a0b40 con 0x7fa594136520 2023-09-29T23:31:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa594067ad0 con 0x7fa594066ec0 2023-09-29T23:31:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa5999d6700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fa59406b680 0x7fa594135e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43210/0 (socket says 172.21.15.143:43210) 2023-09-29T23:31:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa5999d6700 1 -- 172.21.15.143:0/1135616206 learned_addr learned my addr 172.21.15.143:0/1135616206 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:31:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59a1d7700 1 --2- 172.21.15.143:0/1135616206 >> v2:172.21.15.143:3300/0 conn(0x7fa594066ec0 0x7fa59409fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa5991d5700 1 -- 172.21.15.143:0/1135616206 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346426211 0 0) 0x7fa594067290 con 0x7fa59406b680 2023-09-29T23:31:54.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa5991d5700 1 -- 172.21.15.143:0/1135616206 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa57c003570 con 0x7fa59406b680 2023-09-29T23:31:54.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59a1d7700 1 -- 172.21.15.143:0/1135616206 >> v1:172.21.15.156:6789/0 conn(0x7fa594136520 legacy=0x7fa59414a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:54.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59a1d7700 1 -- 172.21.15.143:0/1135616206 >> v1:172.21.15.143:6789/0 conn(0x7fa59406b680 legacy=0x7fa594135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:54.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa5991d5700 1 -- 172.21.15.143:0/1135616206 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2044633025 0 0) 0x7fa5940a0b40 con 0x7fa594136520 2023-09-29T23:31:54.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59a1d7700 1 -- 172.21.15.143:0/1135616206 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5940a0dc0 con 0x7fa594066ec0 2023-09-29T23:31:54.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59a1d7700 1 --2- 172.21.15.143:0/1135616206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa594066ec0 0x7fa59409fc50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa5880098e0 tx=0x7fa5880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e536e693d9dfde63 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:54.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa5991d5700 1 -- 172.21.15.143:0/1135616206 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58800b640 con 0x7fa594066ec0 2023-09-29T23:31:54.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa5991d5700 1 -- 172.21.15.143:0/1135616206 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58800b7a0 con 0x7fa594066ec0 2023-09-29T23:31:54.612 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa5991d5700 1 -- 172.21.15.143:0/1135616206 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58800bb20 con 0x7fa594066ec0 2023-09-29T23:31:54.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 -- 172.21.15.143:0/1135616206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa594066ec0 msgr2=0x7fa59409fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:54.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 --2- 172.21.15.143:0/1135616206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa594066ec0 0x7fa59409fc50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa5880098e0 tx=0x7fa5880091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:54.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 -- 172.21.15.143:0/1135616206 shutdown_connections 2023-09-29T23:31:54.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 --2- 172.21.15.143:0/1135616206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa594066ec0 0x7fa59409fc50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:54.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 -- 172.21.15.143:0/1135616206 >> 172.21.15.143:0/1135616206 conn(0x7fa59406f200 msgr2=0x7fa594069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:54.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 -- 172.21.15.143:0/1135616206 shutdown_connections 2023-09-29T23:31:54.613 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.609+0000 7fa59b1d9700 1 -- 172.21.15.143:0/1135616206 wait complete. 2023-09-29T23:31:54.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.613+0000 7fa59b1d9700 1 Processor -- start 2023-09-29T23:31:54.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.613+0000 7fa59b1d9700 1 -- start start 2023-09-29T23:31:54.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.613+0000 7fa59b1d9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa594066ec0 0x7fa59409b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:54.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.613+0000 7fa59b1d9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59406b680 0x7fa59409c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:54.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.613+0000 7fa59b1d9700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa594136520 0x7fa594153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:54.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.613+0000 7fa59b1d9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59409ca50 con 0x7fa59406b680 2023-09-29T23:31:54.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.613+0000 7fa59b1d9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa594154070 con 0x7fa594066ec0 2023-09-29T23:31:54.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.613+0000 7fa59b1d9700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59409c830 con 0x7fa594136520 2023-09-29T23:31:54.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa5999d6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59406b680 0x7fa59409c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:54.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa59a9d8700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa594136520 0x7fa594153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:54.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa5999d6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59406b680 0x7fa59409c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46316/0 (socket says 172.21.15.143:46316) 2023-09-29T23:31:54.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa59a1d7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa594066ec0 0x7fa59409b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:54.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa5999d6700 1 -- 172.21.15.143:0/3583029239 learned_addr learned my addr 172.21.15.143:0/3583029239 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:54.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa59a9d8700 1 -- 172.21.15.143:0/3583029239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa594066ec0 msgr2=0x7fa59409b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:54.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa59a9d8700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa594066ec0 0x7fa59409b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:54.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa59a9d8700 1 -- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59406b680 msgr2=0x7fa59409c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:54.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa59a9d8700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59406b680 0x7fa59409c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:54.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa59a9d8700 1 -- 172.21.15.143:0/3583029239 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa588009420 con 0x7fa594136520 2023-09-29T23:31:54.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa59a9d8700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa594136520 0x7fa594153160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00c490 tx=0x7fa58c007ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:54.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa5927fc700 1 -- 172.21.15.143:0/3583029239 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00ce00 con 0x7fa594136520 2023-09-29T23:31:54.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa5927fc700 1 -- 172.21.15.143:0/3583029239 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c011590 con 0x7fa594136520 2023-09-29T23:31:54.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa59b1d9700 1 -- 172.21.15.143:0/3583029239 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa594153db0 con 0x7fa594136520 2023-09-29T23:31:54.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa5927fc700 1 -- 172.21.15.143:0/3583029239 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c011950 con 0x7fa594136520 2023-09-29T23:31:54.619 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa59b1d9700 1 -- 172.21.15.143:0/3583029239 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5941536a0 con 0x7fa594136520 2023-09-29T23:31:54.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.617+0000 7fa59b1d9700 1 -- 172.21.15.143:0/3583029239 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59409cc00 con 0x7fa594136520 2023-09-29T23:31:54.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.621+0000 7fa5927fc700 1 -- 172.21.15.143:0/3583029239 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fa58c011d00 con 0x7fa594136520 2023-09-29T23:31:54.620 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.621+0000 7fa5927fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.621+0000 7fa5927fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:54.621 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.621+0000 7fa5927fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:54.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.621+0000 7fa5927fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:54.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.621+0000 7fa5927fc700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa58008e8b0 0x7fa580090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:54.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.621+0000 7fa59a1d7700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa58008e8b0 0x7fa580090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:54.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.621+0000 7fa5927fc700 1 -- 172.21.15.143:0/3583029239 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fa58c0b3ba0 con 0x7fa594136520 2023-09-29T23:31:54.622 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.621+0000 7fa59a1d7700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa58008e8b0 0x7fa580090d60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5880093f0 tx=0x7fa588017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:54.626 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.625+0000 7fa5927fc700 1 -- 172.21.15.143:0/3583029239 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fa59409cc00 con 0x7fa594136520 2023-09-29T23:31:54.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.821+0000 7fa59b1d9700 1 -- 172.21.15.143:0/3583029239 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa59409cc00 con 0x7fa594136520 2023-09-29T23:31:54.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.821+0000 7fa5927fc700 1 -- 172.21.15.143:0/3583029239 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fa59409cc00 con 0x7fa594136520 2023-09-29T23:31:54.821 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:54.821 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 10 mgrc shutdown 2023-09-29T23:31:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 -- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa58008e8b0 msgr2=0x7fa580090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:54.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa58008e8b0 0x7fa580090d60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa5880093f0 tx=0x7fa588017000 comp rx=0 tx=0).stop 2023-09-29T23:31:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 -- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa594136520 msgr2=0x7fa594153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa594136520 0x7fa594153160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa58c00c490 tx=0x7fa58c007ea0 comp rx=0 tx=0).stop 2023-09-29T23:31:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 -- 172.21.15.143:0/3583029239 shutdown_connections 2023-09-29T23:31:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fa58008e8b0 0x7fa580090d60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fa594136520 0x7fa594153160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa59406b680 0x7fa59409c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 --2- 172.21.15.143:0/3583029239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa594066ec0 0x7fa59409b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 -- 172.21.15.143:0/3583029239 >> 172.21.15.143:0/3583029239 conn(0x7fa59406f200 msgr2=0x7fa59406c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 -- 172.21.15.143:0/3583029239 shutdown_connections 2023-09-29T23:31:54.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:54.825+0000 7fa59b1d9700 1 -- 172.21.15.143:0/3583029239 wait complete. 2023-09-29T23:31:56.095 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:31:56.097 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:31:57.840 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:31:57.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 Processor -- start 2023-09-29T23:31:57.953 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 -- start start 2023-09-29T23:31:57.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 0x7fed980a2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:57.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fed980a09f0 0x7fed9813f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:57.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed980a35d0 con 0x7fed980a0dc0 2023-09-29T23:31:57.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed980a2790 con 0x7fed980a09f0 2023-09-29T23:31:57.957 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed980a8c30 con 0x7fed98067e40 2023-09-29T23:31:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed8f7fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fed980a09f0 0x7fed9813f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed97fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 0x7fed980a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9cbf0700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fed980a0dc0 0x7fed98143840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43216/0 (socket says 172.21.15.143:43216) 2023-09-29T23:31:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9cbf0700 1 -- 172.21.15.143:0/3817005501 learned_addr learned my addr 172.21.15.143:0/3817005501 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:31:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed8f7fe700 1 -- 172.21.15.143:0/3817005501 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 msgr2=0x7fed980a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:31:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed8f7fe700 1 --2- 172.21.15.143:0/3817005501 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 0x7fed980a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed8f7fe700 1 -- 172.21.15.143:0/3817005501 >> v1:172.21.15.143:6789/0 conn(0x7fed980a0dc0 legacy=0x7fed98143840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed8f7fe700 1 -- 172.21.15.143:0/3817005501 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed980a8ed0 con 0x7fed980a09f0 2023-09-29T23:31:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed977fe700 1 -- 172.21.15.143:0/3817005501 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73182396 0 0) 0x7fed980a35d0 con 0x7fed980a0dc0 2023-09-29T23:31:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed8f7fe700 1 --2- 172.21.15.143:0/3817005501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed980a09f0 0x7fed9813f8d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fed980a90c0 tx=0x7fed800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca52e758e86919e server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:57.958 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed977fe700 1 -- 172.21.15.143:0/3817005501 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80011630 con 0x7fed980a09f0 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed977fe700 1 -- 172.21.15.143:0/3817005501 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed80011790 con 0x7fed980a09f0 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed977fe700 1 -- 172.21.15.143:0/3817005501 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed80011b50 con 0x7fed980a09f0 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 -- 172.21.15.143:0/3817005501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed980a09f0 msgr2=0x7fed9813f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 --2- 172.21.15.143:0/3817005501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed980a09f0 0x7fed9813f8d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fed980a90c0 tx=0x7fed800091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 -- 172.21.15.143:0/3817005501 shutdown_connections 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 --2- 172.21.15.143:0/3817005501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed980a09f0 0x7fed9813f8d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 --2- 172.21.15.143:0/3817005501 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 0x7fed980a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 -- 172.21.15.143:0/3817005501 >> 172.21.15.143:0/3817005501 conn(0x7fed980b6ac0 msgr2=0x7fed980b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 -- 172.21.15.143:0/3817005501 shutdown_connections 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.949+0000 7fed9e653700 1 -- 172.21.15.143:0/3817005501 wait complete. 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed9e653700 1 Processor -- start 2023-09-29T23:31:57.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed9e653700 1 -- start start 2023-09-29T23:31:57.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed9e653700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 0x7fed9813c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:57.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed9e653700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed980a09f0 0x7fed98143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:57.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed9e653700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed980a0dc0 0x7fed98143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:57.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed97fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 0x7fed9813c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:57.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed9e653700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed98143ba0 con 0x7fed980a0dc0 2023-09-29T23:31:57.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed9e653700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed980a62f0 con 0x7fed980a09f0 2023-09-29T23:31:57.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed9e653700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed98144030 con 0x7fed98067e40 2023-09-29T23:31:57.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed97fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 0x7fed9813c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:49360/0 (socket says 172.21.15.143:49360) 2023-09-29T23:31:57.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed97fff700 1 -- 172.21.15.143:0/769822735 learned_addr learned my addr 172.21.15.143:0/769822735 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:31:57.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.953+0000 7fed9cbf0700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed980a0dc0 0x7fed98143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:57.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed8f7fe700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed980a09f0 0x7fed98143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed97fff700 1 -- 172.21.15.143:0/769822735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed980a09f0 msgr2=0x7fed98143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed97fff700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed980a09f0 0x7fed98143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed97fff700 1 -- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed980a0dc0 msgr2=0x7fed98143660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed97fff700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed980a0dc0 0x7fed98143660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed97fff700 1 -- 172.21.15.143:0/769822735 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed80009420 con 0x7fed98067e40 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed9cbf0700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed980a0dc0 0x7fed98143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed97fff700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 0x7fed9813c210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed8800ecb0 tx=0x7fed880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed94ff9700 1 -- 172.21.15.143:0/769822735 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8800c920 con 0x7fed98067e40 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed94ff9700 1 -- 172.21.15.143:0/769822735 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8800ca80 con 0x7fed98067e40 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed9e653700 1 -- 172.21.15.143:0/769822735 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed98143d90 con 0x7fed98067e40 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed94ff9700 1 -- 172.21.15.143:0/769822735 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8800ce20 con 0x7fed98067e40 2023-09-29T23:31:57.961 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed9e653700 1 -- 172.21.15.143:0/769822735 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9813c750 con 0x7fed98067e40 2023-09-29T23:31:57.962 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.957+0000 7fed9e653700 1 -- 172.21.15.143:0/769822735 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9805ecb0 con 0x7fed98067e40 2023-09-29T23:31:57.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.961+0000 7fed94ff9700 1 -- 172.21.15.143:0/769822735 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fed88013680 con 0x7fed98067e40 2023-09-29T23:31:57.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.961+0000 7fed94ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:31:57.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.961+0000 7fed94ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:31:57.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.961+0000 7fed94ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:57.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.961+0000 7fed94ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:31:57.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.961+0000 7fed94ff9700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fed7808e8b0 0x7fed78090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:31:57.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.961+0000 7fed8f7fe700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fed7808e8b0 0x7fed78090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:31:57.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.961+0000 7fed94ff9700 1 -- 172.21.15.143:0/769822735 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fed880b5300 con 0x7fed98067e40 2023-09-29T23:31:57.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.961+0000 7fed8f7fe700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fed7808e8b0 0x7fed78090d60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fed800091e0 tx=0x7fed80016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:31:57.965 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:57.965+0000 7fed94ff9700 1 -- 172.21.15.143:0/769822735 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fed88085cc0 con 0x7fed98067e40 2023-09-29T23:31:58.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.157+0000 7fed9e653700 1 -- 172.21.15.143:0/769822735 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fed980b6380 con 0x7fed98067e40 2023-09-29T23:31:58.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.157+0000 7fed94ff9700 1 -- 172.21.15.143:0/769822735 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fed88021100 con 0x7fed98067e40 2023-09-29T23:31:58.159 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:31:58.159 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:31:58.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 10 mgrc shutdown 2023-09-29T23:31:58.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 -- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fed7808e8b0 msgr2=0x7fed78090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:58.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fed7808e8b0 0x7fed78090d60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fed800091e0 tx=0x7fed80016000 comp rx=0 tx=0).stop 2023-09-29T23:31:58.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 -- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 msgr2=0x7fed9813c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:31:58.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 0x7fed9813c210 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fed8800ecb0 tx=0x7fed880091a0 comp rx=0 tx=0).stop 2023-09-29T23:31:58.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 -- 172.21.15.143:0/769822735 shutdown_connections 2023-09-29T23:31:58.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fed7808e8b0 0x7fed78090d60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:58.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed980a0dc0 0x7fed98143660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:58.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed980a09f0 0x7fed98143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:58.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 --2- 172.21.15.143:0/769822735 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fed98067e40 0x7fed9813c210 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:31:58.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 -- 172.21.15.143:0/769822735 >> 172.21.15.143:0/769822735 conn(0x7fed980b6ac0 msgr2=0x7fed980b56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:31:58.162 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 -- 172.21.15.143:0/769822735 shutdown_connections 2023-09-29T23:31:58.163 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:31:58.161+0000 7fed9e653700 1 -- 172.21.15.143:0/769822735 wait complete. 2023-09-29T23:32:01.174 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:01.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.281+0000 7f4987a15700 1 Processor -- start 2023-09-29T23:32:01.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.281+0000 7f4987a15700 1 -- start start 2023-09-29T23:32:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f49800a88c0 0x7f49801361e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f498014ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49800a9820 con 0x7f49800675d0 2023-09-29T23:32:01.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49800a8ce0 con 0x7f49800a88c0 2023-09-29T23:32:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49800679a0 con 0x7f4980136720 2023-09-29T23:32:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4985fb2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f498014ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4985fb2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f498014ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:49368/0 (socket says 172.21.15.143:49368) 2023-09-29T23:32:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4985fb2700 1 -- 172.21.15.143:0/3807740226 learned_addr learned my addr 172.21.15.143:0/3807740226 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4984fb0700 1 --2- 172.21.15.143:0/3807740226 >> v2:172.21.15.156:3300/0 conn(0x7f49800a88c0 0x7f49801361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4985fb2700 1 -- 172.21.15.143:0/3807740226 >> v2:172.21.15.156:3300/0 conn(0x7f49800a88c0 msgr2=0x7f49801361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4985fb2700 1 --2- 172.21.15.143:0/3807740226 >> v2:172.21.15.156:3300/0 conn(0x7f49800a88c0 0x7f49801361e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:01.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4985fb2700 1 -- 172.21.15.143:0/3807740226 >> v1:172.21.15.156:6789/0 conn(0x7f49800675d0 legacy=0x7f49800a9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4985fb2700 1 -- 172.21.15.143:0/3807740226 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498006b4e0 con 0x7f4980136720 2023-09-29T23:32:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4985fb2700 1 --2- 172.21.15.143:0/3807740226 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f498014ad50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4978009f20 tx=0x7f4978009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0596395b7d01263 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4977fff700 1 -- 172.21.15.143:0/3807740226 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497800e070 con 0x7f4980136720 2023-09-29T23:32:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4977fff700 1 -- 172.21.15.143:0/3807740226 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497800ba40 con 0x7f4980136720 2023-09-29T23:32:01.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4977fff700 1 -- 172.21.15.143:0/3807740226 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49780092e0 con 0x7f4980136720 2023-09-29T23:32:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 -- 172.21.15.143:0/3807740226 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 msgr2=0x7f498014ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 --2- 172.21.15.143:0/3807740226 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f498014ad50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4978009f20 tx=0x7f4978009840 comp rx=0 tx=0).stop 2023-09-29T23:32:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 -- 172.21.15.143:0/3807740226 shutdown_connections 2023-09-29T23:32:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 --2- 172.21.15.143:0/3807740226 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f498014ad50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 --2- 172.21.15.143:0/3807740226 >> v2:172.21.15.156:3300/0 conn(0x7f49800a88c0 0x7f49801361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:01.290 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 -- 172.21.15.143:0/3807740226 >> 172.21.15.143:0/3807740226 conn(0x7f49800b76e0 msgr2=0x7f4980066c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 -- 172.21.15.143:0/3807740226 shutdown_connections 2023-09-29T23:32:01.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.285+0000 7f4987a15700 1 -- 172.21.15.143:0/3807740226 wait complete. 2023-09-29T23:32:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4987a15700 1 Processor -- start 2023-09-29T23:32:01.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4987a15700 1 -- start start 2023-09-29T23:32:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4987a15700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49800675d0 0x7f49800aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4987a15700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49800a88c0 0x7f49800b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4987a15700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f4980150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4987a15700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800b0b00 con 0x7f49800a88c0 2023-09-29T23:32:01.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4987a15700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800ad7d0 con 0x7f49800675d0 2023-09-29T23:32:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4987a15700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4980150f10 con 0x7f4980136720 2023-09-29T23:32:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4984fb0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49800a88c0 0x7f49800b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4985fb2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f4980150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4985fb2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f4980150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:49374/0 (socket says 172.21.15.143:49374) 2023-09-29T23:32:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4985fb2700 1 -- 172.21.15.143:0/4273621509 learned_addr learned my addr 172.21.15.143:0/4273621509 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:01.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4984fb0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49800a88c0 0x7f49800b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46330/0 (socket says 172.21.15.143:46330) 2023-09-29T23:32:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f49857b1700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49800675d0 0x7f49800aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4985fb2700 1 -- 172.21.15.143:0/4273621509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49800675d0 msgr2=0x7f49800aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4985fb2700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49800675d0 0x7f49800aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4985fb2700 1 -- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49800a88c0 msgr2=0x7f49800b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4985fb2700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49800a88c0 0x7f49800b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.289+0000 7f4985fb2700 1 -- 172.21.15.143:0/4273621509 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4978009ae0 con 0x7f4980136720 2023-09-29T23:32:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.293+0000 7f4985fb2700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f4980150280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4978009880 tx=0x7f4978017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.293+0000 7f49757fa700 1 -- 172.21.15.143:0/4273621509 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497800beb0 con 0x7f4980136720 2023-09-29T23:32:01.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.293+0000 7f49757fa700 1 -- 172.21.15.143:0/4273621509 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49780092e0 con 0x7f4980136720 2023-09-29T23:32:01.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.293+0000 7f4987a15700 1 -- 172.21.15.143:0/4273621509 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49800b3d70 con 0x7f4980136720 2023-09-29T23:32:01.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.293+0000 7f49757fa700 1 -- 172.21.15.143:0/4273621509 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497801e430 con 0x7f4980136720 2023-09-29T23:32:01.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.293+0000 7f4987a15700 1 -- 172.21.15.143:0/4273621509 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4980150960 con 0x7f4980136720 2023-09-29T23:32:01.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.293+0000 7f4987a15700 1 -- 172.21.15.143:0/4273621509 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498005ecb0 con 0x7f4980136720 2023-09-29T23:32:01.298 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.297+0000 7f49757fa700 1 -- 172.21.15.143:0/4273621509 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f497801e860 con 0x7f4980136720 2023-09-29T23:32:01.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.297+0000 7f49757fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:01.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.297+0000 7f49757fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:01.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.297+0000 7f49757fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:01.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.297+0000 7f49757fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:01.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.297+0000 7f49757fa700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f496408e8b0 0x7f4964090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:01.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.297+0000 7f49857b1700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f496408e8b0 0x7f4964090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:01.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.297+0000 7f49757fa700 1 -- 172.21.15.143:0/4273621509 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f49780b6da0 con 0x7f4980136720 2023-09-29T23:32:01.299 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.297+0000 7f49857b1700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f496408e8b0 0x7f4964090d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f497c003a80 tx=0x7f497c007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:01.303 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.301+0000 7f49757fa700 1 -- 172.21.15.143:0/4273621509 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f49780877f0 con 0x7f4980136720 2023-09-29T23:32:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.501+0000 7f4987a15700 1 -- 172.21.15.143:0/4273621509 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f498006b7a0 con 0x7f4980136720 2023-09-29T23:32:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.501+0000 7f49757fa700 1 -- 172.21.15.143:0/4273621509 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f497808ae10 con 0x7f4980136720 2023-09-29T23:32:01.501 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:01.501 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:01.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 10 mgrc shutdown 2023-09-29T23:32:01.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 -- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f496408e8b0 msgr2=0x7f4964090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:01.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f496408e8b0 0x7f4964090d60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f497c003a80 tx=0x7f497c007ec0 comp rx=0 tx=0).stop 2023-09-29T23:32:01.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 -- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 msgr2=0x7f4980150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:01.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f4980150280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4978009880 tx=0x7f4978017000 comp rx=0 tx=0).stop 2023-09-29T23:32:01.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 -- 172.21.15.143:0/4273621509 shutdown_connections 2023-09-29T23:32:01.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f496408e8b0 0x7f4964090d60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:01.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4980136720 0x7f4980150280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:01.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49800a88c0 0x7f49800b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:01.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 --2- 172.21.15.143:0/4273621509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49800675d0 0x7f49800aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:01.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 -- 172.21.15.143:0/4273621509 >> 172.21.15.143:0/4273621509 conn(0x7f49800b76e0 msgr2=0x7f49800b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:01.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 -- 172.21.15.143:0/4273621509 shutdown_connections 2023-09-29T23:32:01.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:01.505+0000 7f4987a15700 1 -- 172.21.15.143:0/4273621509 wait complete. 2023-09-29T23:32:04.521 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:04.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f5462fd0700 1 Processor -- start 2023-09-29T23:32:04.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f5462fd0700 1 -- start start 2023-09-29T23:32:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f5462fd0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f545c0b3140 0x7f545c1442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f5462fd0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f545c136660 0x7f545c143820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f5462fd0700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c143e60 con 0x7f545c06da00 2023-09-29T23:32:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f5462fd0700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c0b3510 con 0x7f545c136660 2023-09-29T23:32:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f5462fd0700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c136a50 con 0x7f545c0b3140 2023-09-29T23:32:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545bfff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f545c06da00 0x7f545c135f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57244/0 (socket says 172.21.15.143:57244) 2023-09-29T23:32:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545bfff700 1 -- 172.21.15.143:0/639281496 learned_addr learned my addr 172.21.15.143:0/639281496 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:32:04.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f546156d700 1 --2- 172.21.15.143:0/639281496 >> v2:172.21.15.143:3300/0 conn(0x7f545c136660 0x7f545c143820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f5460d6c700 1 --2- 172.21.15.143:0/639281496 >> v2:172.21.15.156:3300/0 conn(0x7f545c0b3140 0x7f545c1442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093912016 0 0) 0x7f545c143e60 con 0x7f545c06da00 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5444003570 con 0x7f545c06da00 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4208323581 0 0) 0x7f5444003570 con 0x7f545c06da00 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 >> v2:172.21.15.156:3300/0 conn(0x7f545c0b3140 msgr2=0x7f545c1442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 --2- 172.21.15.143:0/639281496 >> v2:172.21.15.156:3300/0 conn(0x7f545c0b3140 0x7f545c1442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 >> v2:172.21.15.143:3300/0 conn(0x7f545c136660 msgr2=0x7f545c143820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 --2- 172.21.15.143:0/639281496 >> v2:172.21.15.143:3300/0 conn(0x7f545c136660 0x7f545c143820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c06b3b0 con 0x7f545c06da00 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251153714 0 0) 0x7f544c0017f0 con 0x7f545c06da00 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 >> v1:172.21.15.143:6789/0 conn(0x7f545c06da00 legacy=0x7f545c135f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:04.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5444003a60 0x7f5444003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5444004810 0x7f5444004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54440051c0 0x7f544400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0b3510 con 0x7f54440051c0 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c136a50 con 0x7f5444003a60 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544400bfd0 con 0x7f5444004810 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545bfff700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5444004810 0x7f5444004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f5460d6c700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5444003a60 0x7f5444003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f546156d700 1 --2- 172.21.15.143:0/639281496 >> v2:172.21.15.143:3300/0 conn(0x7f545c136660 0x7f545c143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f546156d700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54440051c0 0x7f544400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545bfff700 1 -- 172.21.15.143:0/639281496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5444003a60 msgr2=0x7f5444003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545bfff700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5444003a60 0x7f5444003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545bfff700 1 -- 172.21.15.143:0/639281496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54440051c0 msgr2=0x7f544400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:04.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545bfff700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54440051c0 0x7f544400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f545bfff700 1 -- 172.21.15.143:0/639281496 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5444003570 con 0x7f5444004810 2023-09-29T23:32:04.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f5460d6c700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5444003a60 0x7f5444003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:04.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f546156d700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f54440051c0 0x7f544400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-29T23:32:04.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.633+0000 7f546156d700 1 -- 172.21.15.143:0/639281496 reap_dead start 2023-09-29T23:32:04.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f545bfff700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5444004810 0x7f5444004c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f544c002250 tx=0x7f544c00a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=369eed951d84a765 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:04.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c0070f0 con 0x7f5444004810 2023-09-29T23:32:04.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c007250 con 0x7f5444004810 2023-09-29T23:32:04.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f545b7fe700 1 -- 172.21.15.143:0/639281496 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c007610 con 0x7f5444004810 2023-09-29T23:32:04.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 -- 172.21.15.143:0/639281496 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5444004810 msgr2=0x7f5444004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:04.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5444004810 0x7f5444004c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f544c002250 tx=0x7f544c00a060 comp rx=0 tx=0).stop 2023-09-29T23:32:04.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 -- 172.21.15.143:0/639281496 shutdown_connections 2023-09-29T23:32:04.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 --2- 172.21.15.143:0/639281496 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f5444004810 0x7f5444004c80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.642 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 -- 172.21.15.143:0/639281496 >> 172.21.15.143:0/639281496 conn(0x7f545c067dc0 msgr2=0x7f545c0b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 -- 172.21.15.143:0/639281496 shutdown_connections 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 -- 172.21.15.143:0/639281496 wait complete. 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 Processor -- start 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 -- start start 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5444004810 0x7f545c137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f545c0acc50 0x7f545c0ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f545c14f930 0x7f545c14fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c1508f0 con 0x7f5444004810 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c138ce0 con 0x7f545c14f930 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f5462fd0700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c138070 con 0x7f545c0acc50 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f545bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f545c0acc50 0x7f545c0ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:04.643 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f545bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f545c0acc50 0x7f545c0ad0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:44920/0 (socket says 172.21.15.143:44920) 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.637+0000 7f545bfff700 1 -- 172.21.15.143:0/2057192583 learned_addr learned my addr 172.21.15.143:0/2057192583 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f5460d6c700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5444004810 0x7f545c137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f546156d700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f545c14f930 0x7f545c14fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f545bfff700 1 -- 172.21.15.143:0/2057192583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f545c14f930 msgr2=0x7f545c14fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f545bfff700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f545c14f930 0x7f545c14fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f545bfff700 1 -- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5444004810 msgr2=0x7f545c137130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f545bfff700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5444004810 0x7f545c137130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f545bfff700 1 -- 172.21.15.143:0/2057192583 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c150630 con 0x7f545c0acc50 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f5460d6c700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5444004810 0x7f545c137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f545bfff700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f545c0acc50 0x7f545c0ad0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f544c00af10 tx=0x7f544c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f5458ff9700 1 -- 172.21.15.143:0/2057192583 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c007070 con 0x7f545c0acc50 2023-09-29T23:32:04.644 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f5458ff9700 1 -- 172.21.15.143:0/2057192583 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544c0017b0 con 0x7f545c0acc50 2023-09-29T23:32:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f5462fd0700 1 -- 172.21.15.143:0/2057192583 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c141070 con 0x7f545c0acc50 2023-09-29T23:32:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f5458ff9700 1 -- 172.21.15.143:0/2057192583 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544c007980 con 0x7f545c0acc50 2023-09-29T23:32:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f5462fd0700 1 -- 172.21.15.143:0/2057192583 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c137670 con 0x7f545c0acc50 2023-09-29T23:32:04.645 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.641+0000 7f5462fd0700 1 -- 172.21.15.143:0/2057192583 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c137fd0 con 0x7f545c0acc50 2023-09-29T23:32:04.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.645+0000 7f5458ff9700 1 -- 172.21.15.143:0/2057192583 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f544c030430 con 0x7f545c0acc50 2023-09-29T23:32:04.646 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.645+0000 7f5458ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:04.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.645+0000 7f5458ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:04.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.645+0000 7f5458ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:04.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.645+0000 7f5458ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:04.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.645+0000 7f5458ff9700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f544808e8b0 0x7f5448090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:04.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.645+0000 7f5460d6c700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f544808e8b0 0x7f5448090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:04.647 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.645+0000 7f5458ff9700 1 -- 172.21.15.143:0/2057192583 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f544c0b6fe0 con 0x7f545c0acc50 2023-09-29T23:32:04.648 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.645+0000 7f5460d6c700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f544808e8b0 0x7f5448090d60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5450006f00 tx=0x7f54500091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:04.651 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.649+0000 7f5458ff9700 1 -- 172.21.15.143:0/2057192583 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f544c0879b0 con 0x7f545c0acc50 2023-09-29T23:32:04.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.845+0000 7f5462fd0700 1 -- 172.21.15.143:0/2057192583 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f545c05ecb0 con 0x7f545c0acc50 2023-09-29T23:32:04.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.845+0000 7f5458ff9700 1 -- 172.21.15.143:0/2057192583 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f544c0075d0 con 0x7f545c0acc50 2023-09-29T23:32:04.845 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:04.845 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:04.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 10 mgrc shutdown 2023-09-29T23:32:04.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 -- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f544808e8b0 msgr2=0x7f5448090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:04.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f544808e8b0 0x7f5448090d60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5450006f00 tx=0x7f54500091a0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 -- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f545c0acc50 msgr2=0x7f545c0ad0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:04.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f545c0acc50 0x7f545c0ad0a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f544c00af10 tx=0x7f544c016000 comp rx=0 tx=0).stop 2023-09-29T23:32:04.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 -- 172.21.15.143:0/2057192583 shutdown_connections 2023-09-29T23:32:04.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f544808e8b0 0x7f5448090d60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f545c14f930 0x7f545c14fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f545c0acc50 0x7f545c0ad0a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 --2- 172.21.15.143:0/2057192583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5444004810 0x7f545c137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:04.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 -- 172.21.15.143:0/2057192583 >> 172.21.15.143:0/2057192583 conn(0x7f545c067dc0 msgr2=0x7f545c0b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:04.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 -- 172.21.15.143:0/2057192583 shutdown_connections 2023-09-29T23:32:04.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:04.849+0000 7f5462fd0700 1 -- 172.21.15.143:0/2057192583 wait complete. 2023-09-29T23:32:07.866 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.973+0000 7f44e6600700 1 Processor -- start 2023-09-29T23:32:07.976 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.973+0000 7f44e6600700 1 -- start start 2023-09-29T23:32:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44e0067e40 0x7f44e00a2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f44e00a0df0 0x7f44e014ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e00a35d0 con 0x7f44e00a09f0 2023-09-29T23:32:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e00a2790 con 0x7f44e0067e40 2023-09-29T23:32:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e00a8c30 con 0x7f44e00a0df0 2023-09-29T23:32:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44dffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44e0067e40 0x7f44e00a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44dffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f44e0067e40 0x7f44e00a2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46466/0 (socket says 172.21.15.143:46466) 2023-09-29T23:32:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44dffff700 1 -- 172.21.15.143:0/1994433539 learned_addr learned my addr 172.21.15.143:0/1994433539 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:07.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e4b9d700 1 --2- 172.21.15.143:0/1994433539 >> v2:172.21.15.156:3300/0 conn(0x7f44e00a0df0 0x7f44e014ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44dffff700 1 -- 172.21.15.143:0/1994433539 >> v2:172.21.15.156:3300/0 conn(0x7f44e00a0df0 msgr2=0x7f44e014ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44dffff700 1 --2- 172.21.15.143:0/1994433539 >> v2:172.21.15.156:3300/0 conn(0x7f44e00a0df0 0x7f44e014ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44dffff700 1 -- 172.21.15.143:0/1994433539 >> v1:172.21.15.156:6789/0 conn(0x7f44e00a09f0 legacy=0x7f44e013f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44dffff700 1 -- 172.21.15.143:0/1994433539 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e00a8ed0 con 0x7f44e0067e40 2023-09-29T23:32:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44dffff700 1 --2- 172.21.15.143:0/1994433539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44e0067e40 0x7f44e00a2250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f44e00a90c0 tx=0x7f44d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1d967f04934bdc server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44deffd700 1 -- 172.21.15.143:0/1994433539 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d800e070 con 0x7f44e0067e40 2023-09-29T23:32:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44deffd700 1 -- 172.21.15.143:0/1994433539 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d800ba10 con 0x7f44e0067e40 2023-09-29T23:32:07.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44deffd700 1 -- 172.21.15.143:0/1994433539 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d80092e0 con 0x7f44e0067e40 2023-09-29T23:32:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 -- 172.21.15.143:0/1994433539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44e0067e40 msgr2=0x7f44e00a2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 --2- 172.21.15.143:0/1994433539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44e0067e40 0x7f44e00a2250 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f44e00a90c0 tx=0x7f44d8009840 comp rx=0 tx=0).stop 2023-09-29T23:32:07.980 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 -- 172.21.15.143:0/1994433539 shutdown_connections 2023-09-29T23:32:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 --2- 172.21.15.143:0/1994433539 >> v2:172.21.15.156:3300/0 conn(0x7f44e00a0df0 0x7f44e014ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 --2- 172.21.15.143:0/1994433539 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44e0067e40 0x7f44e00a2250 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 -- 172.21.15.143:0/1994433539 >> 172.21.15.143:0/1994433539 conn(0x7f44e00b6ac0 msgr2=0x7f44e00b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 -- 172.21.15.143:0/1994433539 shutdown_connections 2023-09-29T23:32:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.977+0000 7f44e6600700 1 -- 172.21.15.143:0/1994433539 wait complete. 2023-09-29T23:32:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e6600700 1 Processor -- start 2023-09-29T23:32:07.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e6600700 1 -- start start 2023-09-29T23:32:07.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e6600700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44e0067e40 0x7f44e013cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:07.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e6600700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f44e00a09f0 0x7f44e009cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:07.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e6600700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44e00a0df0 0x7f44e009d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e6600700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e009c6c0 con 0x7f44e00a0df0 2023-09-29T23:32:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e6600700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e006d050 con 0x7f44e0067e40 2023-09-29T23:32:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e6600700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e006d310 con 0x7f44e00a09f0 2023-09-29T23:32:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e4b9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44e00a0df0 0x7f44e009d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44dffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44e0067e40 0x7f44e013cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e4b9d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44e00a0df0 0x7f44e009d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:46470/0 (socket says 172.21.15.143:46470) 2023-09-29T23:32:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e4b9d700 1 -- 172.21.15.143:0/690340237 learned_addr learned my addr 172.21.15.143:0/690340237 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:07.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44df7fe700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f44e00a09f0 0x7f44e009cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e4b9d700 1 -- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f44e00a09f0 msgr2=0x7f44e009cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e4b9d700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f44e00a09f0 0x7f44e009cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e4b9d700 1 -- 172.21.15.143:0/690340237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44e0067e40 msgr2=0x7f44e013cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e4b9d700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44e0067e40 0x7f44e013cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e4b9d700 1 -- 172.21.15.143:0/690340237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d8009ae0 con 0x7f44e00a0df0 2023-09-29T23:32:07.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44df7fe700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f44e00a09f0 0x7f44e009cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:32:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44e4b9d700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44e00a0df0 0x7f44e009d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f44d000d860 tx=0x7f44d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.981+0000 7f44d7fff700 1 -- 172.21.15.143:0/690340237 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d00125f0 con 0x7f44e00a0df0 2023-09-29T23:32:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.985+0000 7f44d7fff700 1 -- 172.21.15.143:0/690340237 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d0012770 con 0x7f44e00a0df0 2023-09-29T23:32:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.985+0000 7f44e6600700 1 -- 172.21.15.143:0/690340237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e006cca0 con 0x7f44e00a0df0 2023-09-29T23:32:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.985+0000 7f44d7fff700 1 -- 172.21.15.143:0/690340237 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d0012b30 con 0x7f44e00a0df0 2023-09-29T23:32:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.985+0000 7f44e6600700 1 -- 172.21.15.143:0/690340237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e013ca80 con 0x7f44e00a0df0 2023-09-29T23:32:07.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.985+0000 7f44e6600700 1 -- 172.21.15.143:0/690340237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e013cc10 con 0x7f44e00a0df0 2023-09-29T23:32:07.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.985+0000 7f44d7fff700 1 -- 172.21.15.143:0/690340237 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f44d001e460 con 0x7f44e00a0df0 2023-09-29T23:32:07.987 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.985+0000 7f44d7fff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.989+0000 7f44d7fff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:07.988 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.989+0000 7f44d7fff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:07.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.989+0000 7f44d7fff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:07.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.989+0000 7f44d7fff700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f44c008e8b0 0x7f44c0090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:07.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.989+0000 7f44dffff700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f44c008e8b0 0x7f44c0090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:07.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.989+0000 7f44d7fff700 1 -- 172.21.15.143:0/690340237 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f44d00b4dd0 con 0x7f44e00a0df0 2023-09-29T23:32:07.989 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.989+0000 7f44dffff700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f44c008e8b0 0x7f44c0090d60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009880 tx=0x7f44d8018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:07.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:07.993+0000 7f44d7fff700 1 -- 172.21.15.143:0/690340237 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f44d0085820 con 0x7f44e00a0df0 2023-09-29T23:32:08.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.189+0000 7f44e6600700 1 -- 172.21.15.143:0/690340237 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f44e005ecb0 con 0x7f44e00a0df0 2023-09-29T23:32:08.218 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.189+0000 7f44d7fff700 1 -- 172.21.15.143:0/690340237 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f44d00b6080 con 0x7f44e00a0df0 2023-09-29T23:32:08.218 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:08.218 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:08.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 10 mgrc shutdown 2023-09-29T23:32:08.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 -- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f44c008e8b0 msgr2=0x7f44c0090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:08.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f44c008e8b0 0x7f44c0090d60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f44d8009880 tx=0x7f44d8018000 comp rx=0 tx=0).stop 2023-09-29T23:32:08.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 -- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44e00a0df0 msgr2=0x7f44e009d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:08.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44e00a0df0 0x7f44e009d300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f44d000d860 tx=0x7f44d0008040 comp rx=0 tx=0).stop 2023-09-29T23:32:08.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 -- 172.21.15.143:0/690340237 shutdown_connections 2023-09-29T23:32:08.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f44c008e8b0 0x7f44c0090d60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:08.220 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44e00a0df0 0x7f44e009d300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:08.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f44e00a09f0 0x7f44e009cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:08.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 --2- 172.21.15.143:0/690340237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44e0067e40 0x7f44e013cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:08.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 -- 172.21.15.143:0/690340237 >> 172.21.15.143:0/690340237 conn(0x7f44e00b6ac0 msgr2=0x7f44e00673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:08.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 -- 172.21.15.143:0/690340237 shutdown_connections 2023-09-29T23:32:08.221 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:08.193+0000 7f44e6600700 1 -- 172.21.15.143:0/690340237 wait complete. 2023-09-29T23:32:11.222 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:11.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 Processor -- start 2023-09-29T23:32:11.337 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 -- start start 2023-09-29T23:32:11.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd3813fce0 0x7fcd380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:11.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcd380b71e0 0x7fcd3814aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:11.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd380a48e0 con 0x7fcd380a5310 2023-09-29T23:32:11.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd380b7640 con 0x7fcd380b71e0 2023-09-29T23:32:11.339 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380a3de0 con 0x7fcd3813fce0 2023-09-29T23:32:11.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3cf25700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd3813fce0 0x7fcd380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:11.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3df27700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fcd380b71e0 0x7fcd3814aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:11.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3cf25700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd3813fce0 0x7fcd380a3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:44934/0 (socket says 172.21.15.143:44934) 2023-09-29T23:32:11.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3cf25700 1 -- 172.21.15.143:0/1585910985 learned_addr learned my addr 172.21.15.143:0/1585910985 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:11.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3df27700 1 -- 172.21.15.143:0/1585910985 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd3813fce0 msgr2=0x7fcd380a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:11.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3df27700 1 --2- 172.21.15.143:0/1585910985 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd3813fce0 0x7fcd380a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:11.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3df27700 1 -- 172.21.15.143:0/1585910985 >> v1:172.21.15.143:6789/0 conn(0x7fcd380a5310 legacy=0x7fcd3813f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:11.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3df27700 1 -- 172.21.15.143:0/1585910985 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd380a40a0 con 0x7fcd380b71e0 2023-09-29T23:32:11.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3df27700 1 --2- 172.21.15.143:0/1585910985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd380b71e0 0x7fcd3814aa10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009e60 tx=0x7fcd30009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8050f790897153 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:11.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd2ffff700 1 -- 172.21.15.143:0/1585910985 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3000e070 con 0x7fcd380b71e0 2023-09-29T23:32:11.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd2ffff700 1 -- 172.21.15.143:0/1585910985 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3000bb60 con 0x7fcd380b71e0 2023-09-29T23:32:11.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd2ffff700 1 -- 172.21.15.143:0/1585910985 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30012430 con 0x7fcd380b71e0 2023-09-29T23:32:11.342 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 -- 172.21.15.143:0/1585910985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd380b71e0 msgr2=0x7fcd3814aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 --2- 172.21.15.143:0/1585910985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd380b71e0 0x7fcd3814aa10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009e60 tx=0x7fcd30009780 comp rx=0 tx=0).stop 2023-09-29T23:32:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 -- 172.21.15.143:0/1585910985 shutdown_connections 2023-09-29T23:32:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 --2- 172.21.15.143:0/1585910985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd380b71e0 0x7fcd3814aa10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 --2- 172.21.15.143:0/1585910985 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd3813fce0 0x7fcd380a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 -- 172.21.15.143:0/1585910985 >> 172.21.15.143:0/1585910985 conn(0x7fcd38066c90 msgr2=0x7fcd38067090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.337+0000 7fcd3e728700 1 -- 172.21.15.143:0/1585910985 shutdown_connections 2023-09-29T23:32:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3e728700 1 -- 172.21.15.143:0/1585910985 wait complete. 2023-09-29T23:32:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3e728700 1 Processor -- start 2023-09-29T23:32:11.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3e728700 1 -- start start 2023-09-29T23:32:11.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3e728700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd380a5310 0x7fcd38143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:11.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3e728700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd380b71e0 0x7fcd3806aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:11.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3e728700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3813fce0 0x7fcd3813c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:11.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3e728700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3809a8c0 con 0x7fcd380a5310 2023-09-29T23:32:11.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3e728700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3806b440 con 0x7fcd3813fce0 2023-09-29T23:32:11.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3e728700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3806b700 con 0x7fcd380b71e0 2023-09-29T23:32:11.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3cf25700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd380b71e0 0x7fcd3806aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:11.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3d726700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd380a5310 0x7fcd38143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3cf25700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd380b71e0 0x7fcd3806aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:44950/0 (socket says 172.21.15.143:44950) 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3cf25700 1 -- 172.21.15.143:0/2180857482 learned_addr learned my addr 172.21.15.143:0/2180857482 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3df27700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3813fce0 0x7fcd3813c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3cf25700 1 -- 172.21.15.143:0/2180857482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3813fce0 msgr2=0x7fcd3813c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.341+0000 7fcd3cf25700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3813fce0 0x7fcd3813c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.345+0000 7fcd3cf25700 1 -- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd380a5310 msgr2=0x7fcd38143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.345+0000 7fcd3cf25700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd380a5310 0x7fcd38143c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.345+0000 7fcd3cf25700 1 -- 172.21.15.143:0/2180857482 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd30009a20 con 0x7fcd380b71e0 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.345+0000 7fcd3d726700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd380a5310 0x7fcd38143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.345+0000 7fcd3cf25700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd380b71e0 0x7fcd3806aa50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400e900 tx=0x7fcd34009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.345+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/2180857482 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34013680 con 0x7fcd380b71e0 2023-09-29T23:32:11.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.345+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/2180857482 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd340137e0 con 0x7fcd380b71e0 2023-09-29T23:32:11.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.345+0000 7fcd3e728700 1 -- 172.21.15.143:0/2180857482 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3809ad50 con 0x7fcd380b71e0 2023-09-29T23:32:11.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.345+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/2180857482 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34013b80 con 0x7fcd380b71e0 2023-09-29T23:32:11.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.345+0000 7fcd3e728700 1 -- 172.21.15.143:0/2180857482 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd380a32b0 con 0x7fcd380b71e0 2023-09-29T23:32:11.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.345+0000 7fcd3e728700 1 -- 172.21.15.143:0/2180857482 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3809aa30 con 0x7fcd380b71e0 2023-09-29T23:32:11.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.349+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/2180857482 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fcd3400c410 con 0x7fcd380b71e0 2023-09-29T23:32:11.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.349+0000 7fcd2d7fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:11.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.349+0000 7fcd2d7fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:11.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.349+0000 7fcd2d7fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:11.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.349+0000 7fcd2d7fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:11.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.349+0000 7fcd2d7fa700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcd1c08e830 0x7fcd1c090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:11.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.349+0000 7fcd3d726700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcd1c08e830 0x7fcd1c090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:11.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.349+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/2180857482 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fcd340b49f0 con 0x7fcd380b71e0 2023-09-29T23:32:11.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.349+0000 7fcd3d726700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcd1c08e830 0x7fcd1c090ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcd380a4a70 tx=0x7fcd28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:11.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.353+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/2180857482 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fcd34088e40 con 0x7fcd380b71e0 2023-09-29T23:32:11.550 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.549+0000 7fcd3e728700 1 -- 172.21.15.143:0/2180857482 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fcd3805ecb0 con 0x7fcd380b71e0 2023-09-29T23:32:11.551 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.549+0000 7fcd2d7fa700 1 -- 172.21.15.143:0/2180857482 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fcd34021100 con 0x7fcd380b71e0 2023-09-29T23:32:11.551 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:11.551 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 10 mgrc shutdown 2023-09-29T23:32:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 -- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcd1c08e830 msgr2=0x7fcd1c090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcd1c08e830 0x7fcd1c090ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcd380a4a70 tx=0x7fcd28009040 comp rx=0 tx=0).stop 2023-09-29T23:32:11.555 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 -- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd380b71e0 msgr2=0x7fcd3806aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:11.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd380b71e0 0x7fcd3806aa50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400e900 tx=0x7fcd34009250 comp rx=0 tx=0).stop 2023-09-29T23:32:11.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 -- 172.21.15.143:0/2180857482 shutdown_connections 2023-09-29T23:32:11.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fcd1c08e830 0x7fcd1c090ce0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:11.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3813fce0 0x7fcd3813c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:11.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fcd380b71e0 0x7fcd3806aa50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:11.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 --2- 172.21.15.143:0/2180857482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcd380a5310 0x7fcd38143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:11.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 -- 172.21.15.143:0/2180857482 >> 172.21.15.143:0/2180857482 conn(0x7fcd38066c90 msgr2=0x7fcd38067b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:11.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 -- 172.21.15.143:0/2180857482 shutdown_connections 2023-09-29T23:32:11.556 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:11.553+0000 7fcd3e728700 1 -- 172.21.15.143:0/2180857482 wait complete. 2023-09-29T23:32:14.571 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:14.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca0d320700 1 Processor -- start 2023-09-29T23:32:14.680 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca0d320700 1 -- start start 2023-09-29T23:32:14.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca0d320700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fca08067e40 0x7fca080a2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:14.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca0d320700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fca080a09f0 0x7fca0813f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:14.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca0d320700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a0df0 0x7fca08143840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:14.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca0d320700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca080a35d0 con 0x7fca080a09f0 2023-09-29T23:32:14.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca0d320700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca080a2790 con 0x7fca08067e40 2023-09-29T23:32:14.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca0d320700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca080a8c30 con 0x7fca080a0df0 2023-09-29T23:32:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca077fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a0df0 0x7fca08143840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca067fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fca080a09f0 0x7fca0813f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca067fc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fca080a09f0 0x7fca0813f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41938/0 (socket says 172.21.15.143:41938) 2023-09-29T23:32:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca077fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a0df0 0x7fca08143840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43314/0 (socket says 172.21.15.143:43314) 2023-09-29T23:32:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca067fc700 1 -- 172.21.15.143:0/2945101901 learned_addr learned my addr 172.21.15.143:0/2945101901 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca06ffd700 1 --2- 172.21.15.143:0/2945101901 >> v2:172.21.15.156:3300/0 conn(0x7fca08067e40 0x7fca080a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca077fe700 1 -- 172.21.15.143:0/2945101901 >> v2:172.21.15.156:3300/0 conn(0x7fca08067e40 msgr2=0x7fca080a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca077fe700 1 --2- 172.21.15.143:0/2945101901 >> v2:172.21.15.156:3300/0 conn(0x7fca08067e40 0x7fca080a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca077fe700 1 -- 172.21.15.143:0/2945101901 >> v2:172.21.15.143:3300/0 conn(0x7fca080a09f0 msgr2=0x7fca0813f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:14.684 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca077fe700 1 --2- 172.21.15.143:0/2945101901 >> v2:172.21.15.143:3300/0 conn(0x7fca080a09f0 0x7fca0813f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca077fe700 1 -- 172.21.15.143:0/2945101901 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca080a8ed0 con 0x7fca080a0df0 2023-09-29T23:32:14.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca067fc700 1 --2- 172.21.15.143:0/2945101901 >> v2:172.21.15.143:3300/0 conn(0x7fca080a09f0 0x7fca0813f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:14.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca077fe700 1 --2- 172.21.15.143:0/2945101901 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a0df0 0x7fca08143840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca080a6c20 tx=0x7fca00009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afe7753e49ec736 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:14.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca05ffb700 1 -- 172.21.15.143:0/2945101901 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0000e070 con 0x7fca080a0df0 2023-09-29T23:32:14.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca05ffb700 1 -- 172.21.15.143:0/2945101901 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca0000bab0 con 0x7fca080a0df0 2023-09-29T23:32:14.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.681+0000 7fca05ffb700 1 -- 172.21.15.143:0/2945101901 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0000be70 con 0x7fca080a0df0 2023-09-29T23:32:14.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.685+0000 7fca0d320700 1 -- 172.21.15.143:0/2945101901 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a0df0 msgr2=0x7fca08143840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:14.686 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.685+0000 7fca0d320700 1 --2- 172.21.15.143:0/2945101901 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a0df0 0x7fca08143840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fca080a6c20 tx=0x7fca00009780 comp rx=0 tx=0).stop 2023-09-29T23:32:14.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.685+0000 7fca0d320700 1 -- 172.21.15.143:0/2945101901 shutdown_connections 2023-09-29T23:32:14.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.685+0000 7fca0d320700 1 --2- 172.21.15.143:0/2945101901 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a0df0 0x7fca08143840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.685+0000 7fca0d320700 1 --2- 172.21.15.143:0/2945101901 >> v2:172.21.15.143:3300/0 conn(0x7fca080a09f0 0x7fca0813f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.685+0000 7fca0d320700 1 --2- 172.21.15.143:0/2945101901 >> v2:172.21.15.156:3300/0 conn(0x7fca08067e40 0x7fca080a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.685+0000 7fca0d320700 1 -- 172.21.15.143:0/2945101901 >> 172.21.15.143:0/2945101901 conn(0x7fca080b6ac0 msgr2=0x7fca080b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:14.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.685+0000 7fca0d320700 1 -- 172.21.15.143:0/2945101901 shutdown_connections 2023-09-29T23:32:14.687 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.685+0000 7fca0d320700 1 -- 172.21.15.143:0/2945101901 wait complete. 2023-09-29T23:32:14.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.685+0000 7fca0d320700 1 Processor -- start 2023-09-29T23:32:14.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.685+0000 7fca0d320700 1 -- start start 2023-09-29T23:32:14.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca0d320700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca08067e40 0x7fca0806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:14.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca0d320700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a09f0 0x7fca080a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:14.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca0d320700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca080a0df0 0x7fca0814ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:14.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca0d320700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0806ab50 con 0x7fca08067e40 2023-09-29T23:32:14.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca0d320700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0806b380 con 0x7fca080a0df0 2023-09-29T23:32:14.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca0d320700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca0806c4b0 con 0x7fca080a09f0 2023-09-29T23:32:14.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca06ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca08067e40 0x7fca0806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:14.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca067fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a09f0 0x7fca080a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:14.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca067fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a09f0 0x7fca080a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43324/0 (socket says 172.21.15.143:43324) 2023-09-29T23:32:14.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca06ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca08067e40 0x7fca0806bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41954/0 (socket says 172.21.15.143:41954) 2023-09-29T23:32:14.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca067fc700 1 -- 172.21.15.143:0/3297448021 learned_addr learned my addr 172.21.15.143:0/3297448021 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:14.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca077fe700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca080a0df0 0x7fca0814ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:14.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca067fc700 1 -- 172.21.15.143:0/3297448021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca080a0df0 msgr2=0x7fca0814ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:14.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca067fc700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca080a0df0 0x7fca0814ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca067fc700 1 -- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca08067e40 msgr2=0x7fca0806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:14.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca067fc700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca08067e40 0x7fca0806bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.691 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca067fc700 1 -- 172.21.15.143:0/3297448021 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f00094f0 con 0x7fca080a09f0 2023-09-29T23:32:14.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca06ffd700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca08067e40 0x7fca0806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:14.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca067fc700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a09f0 0x7fca080a5560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000ec80 tx=0x7fc9f0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:14.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fc9feffd700 1 -- 172.21.15.143:0/3297448021 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000c850 con 0x7fca080a09f0 2023-09-29T23:32:14.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fc9feffd700 1 -- 172.21.15.143:0/3297448021 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f000c9b0 con 0x7fca080a09f0 2023-09-29T23:32:14.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca0d320700 1 -- 172.21.15.143:0/3297448021 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca00009a20 con 0x7fca080a09f0 2023-09-29T23:32:14.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fc9feffd700 1 -- 172.21.15.143:0/3297448021 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000cd50 con 0x7fca080a09f0 2023-09-29T23:32:14.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca0d320700 1 -- 172.21.15.143:0/3297448021 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca081430e0 con 0x7fca080a09f0 2023-09-29T23:32:14.692 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.689+0000 7fca0d320700 1 -- 172.21.15.143:0/3297448021 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca08142ea0 con 0x7fca080a09f0 2023-09-29T23:32:14.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.693+0000 7fc9feffd700 1 -- 172.21.15.143:0/3297448021 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fc9f0013680 con 0x7fca080a09f0 2023-09-29T23:32:14.695 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.693+0000 7fc9feffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:14.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.693+0000 7fc9feffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:14.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.693+0000 7fc9feffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:14.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.693+0000 7fc9feffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:14.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.693+0000 7fc9feffd700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc9ec08e8b0 0x7fc9ec090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:14.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.693+0000 7fc9feffd700 1 -- 172.21.15.143:0/3297448021 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fc9f00b4e80 con 0x7fca080a09f0 2023-09-29T23:32:14.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.693+0000 7fca06ffd700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc9ec08e8b0 0x7fc9ec090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:14.696 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.697+0000 7fca06ffd700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc9ec08e8b0 0x7fc9ec090d60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8011480 tx=0x7fc9f80091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:14.701 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.701+0000 7fc9feffd700 1 -- 172.21.15.143:0/3297448021 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc9f00858d0 con 0x7fca080a09f0 2023-09-29T23:32:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.893+0000 7fca0d320700 1 -- 172.21.15.143:0/3297448021 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fca0814fd10 con 0x7fca080a09f0 2023-09-29T23:32:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.893+0000 7fc9feffd700 1 -- 172.21.15.143:0/3297448021 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fc9f0021100 con 0x7fca080a09f0 2023-09-29T23:32:14.896 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:14.896 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:14.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 10 mgrc shutdown 2023-09-29T23:32:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 -- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc9ec08e8b0 msgr2=0x7fc9ec090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc9ec08e8b0 0x7fc9ec090d60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8011480 tx=0x7fc9f80091a0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 -- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a09f0 msgr2=0x7fca080a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a09f0 0x7fca080a5560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc9f000ec80 tx=0x7fc9f0009250 comp rx=0 tx=0).stop 2023-09-29T23:32:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 -- 172.21.15.143:0/3297448021 shutdown_connections 2023-09-29T23:32:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc9ec08e8b0 0x7fc9ec090d60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca080a0df0 0x7fca0814ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fca080a09f0 0x7fca080a5560 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 --2- 172.21.15.143:0/3297448021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca08067e40 0x7fca0806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 -- 172.21.15.143:0/3297448021 >> 172.21.15.143:0/3297448021 conn(0x7fca080b6ac0 msgr2=0x7fca080b6e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:14.900 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 -- 172.21.15.143:0/3297448021 shutdown_connections 2023-09-29T23:32:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:14.897+0000 7fca0d320700 1 -- 172.21.15.143:0/3297448021 wait complete. 2023-09-29T23:32:17.915 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:18.030 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f62a0ec1700 1 Processor -- start 2023-09-29T23:32:18.031 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f62a0ec1700 1 -- start start 2023-09-29T23:32:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f62a0ec1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c066ec0 0x7f629c06c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f62a0ec1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f629c06e930 0x7f629c0a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f62a0ec1700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f629c06cd10 con 0x7f629c13c010 2023-09-29T23:32:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f62a0ec1700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f629c067290 con 0x7f629c06e930 2023-09-29T23:32:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f62a0ec1700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c06ad80 con 0x7f629c066ec0 2023-09-29T23:32:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f629b7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c066ec0 0x7f629c06c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f629b7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c066ec0 0x7f629c06c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43336/0 (socket says 172.21.15.143:43336) 2023-09-29T23:32:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f629b7fe700 1 -- 172.21.15.143:0/1870341244 learned_addr learned my addr 172.21.15.143:0/1870341244 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f629affd700 1 --2- 172.21.15.143:0/1870341244 >> v2:172.21.15.143:3300/0 conn(0x7f629c06e930 0x7f629c0a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:18.032 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f629b7fe700 1 -- 172.21.15.143:0/1870341244 >> v2:172.21.15.143:3300/0 conn(0x7f629c06e930 msgr2=0x7f629c0a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:18.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f629b7fe700 1 --2- 172.21.15.143:0/1870341244 >> v2:172.21.15.143:3300/0 conn(0x7f629c06e930 0x7f629c0a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:18.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f629b7fe700 1 -- 172.21.15.143:0/1870341244 >> v1:172.21.15.156:6789/0 conn(0x7f629c13c010 legacy=0x7f629c13c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:18.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.028+0000 7f629b7fe700 1 -- 172.21.15.143:0/1870341244 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f629c06b040 con 0x7f629c066ec0 2023-09-29T23:32:18.033 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629b7fe700 1 --2- 172.21.15.143:0/1870341244 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c066ec0 0x7f629c06c7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f628c0099b0 tx=0x7f628c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f69a50fe401aaf8 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:18.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629a7fc700 1 -- 172.21.15.143:0/1870341244 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c009ea0 con 0x7f629c066ec0 2023-09-29T23:32:18.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629a7fc700 1 -- 172.21.15.143:0/1870341244 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f628c00da30 con 0x7f629c066ec0 2023-09-29T23:32:18.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629a7fc700 1 -- 172.21.15.143:0/1870341244 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f628c00ddd0 con 0x7f629c066ec0 2023-09-29T23:32:18.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 -- 172.21.15.143:0/1870341244 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c066ec0 msgr2=0x7f629c06c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:18.034 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 --2- 172.21.15.143:0/1870341244 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c066ec0 0x7f629c06c7d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f628c0099b0 tx=0x7f628c009250 comp rx=0 tx=0).stop 2023-09-29T23:32:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 -- 172.21.15.143:0/1870341244 shutdown_connections 2023-09-29T23:32:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 --2- 172.21.15.143:0/1870341244 >> v2:172.21.15.143:3300/0 conn(0x7f629c06e930 0x7f629c0a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 --2- 172.21.15.143:0/1870341244 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c066ec0 0x7f629c06c7d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 -- 172.21.15.143:0/1870341244 >> 172.21.15.143:0/1870341244 conn(0x7f629c06f200 msgr2=0x7f629c069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 -- 172.21.15.143:0/1870341244 shutdown_connections 2023-09-29T23:32:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 -- 172.21.15.143:0/1870341244 wait complete. 2023-09-29T23:32:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 Processor -- start 2023-09-29T23:32:18.035 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 -- start start 2023-09-29T23:32:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f629c066ec0 0x7f629c1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c06e930 0x7f629c09c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c13c010 0x7f629c09b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c0a43b0 con 0x7f629c06e930 2023-09-29T23:32:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629c150ab0 con 0x7f629c066ec0 2023-09-29T23:32:18.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f62a0ec1700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629c150d50 con 0x7f629c13c010 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c13c010 0x7f629c09b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c06e930 0x7f629c09c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c13c010 0x7f629c09b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43348/0 (socket says 172.21.15.143:43348) 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629affd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c06e930 0x7f629c09c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41966/0 (socket says 172.21.15.143:41966) 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629bfff700 1 -- 172.21.15.143:0/3814078316 learned_addr learned my addr 172.21.15.143:0/3814078316 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629b7fe700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f629c066ec0 0x7f629c1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629affd700 1 -- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c13c010 msgr2=0x7f629c09b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629affd700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c13c010 0x7f629c09b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629affd700 1 -- 172.21.15.143:0/3814078316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f629c066ec0 msgr2=0x7f629c1500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629affd700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f629c066ec0 0x7f629c1500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629affd700 1 -- 172.21.15.143:0/3814078316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f628c0094f0 con 0x7f629c06e930 2023-09-29T23:32:18.038 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629b7fe700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f629c066ec0 0x7f629c1500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:32:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.032+0000 7f629bfff700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c13c010 0x7f629c09b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.036+0000 7f629affd700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c06e930 0x7f629c09c770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f629000ea50 tx=0x7f62900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.036+0000 7f62837fe700 1 -- 172.21.15.143:0/3814078316 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629000c7f0 con 0x7f629c06e930 2023-09-29T23:32:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.036+0000 7f62837fe700 1 -- 172.21.15.143:0/3814078316 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629000c970 con 0x7f629c06e930 2023-09-29T23:32:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.036+0000 7f62a0ec1700 1 -- 172.21.15.143:0/3814078316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f629c09bf90 con 0x7f629c06e930 2023-09-29T23:32:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.036+0000 7f62837fe700 1 -- 172.21.15.143:0/3814078316 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629000cd30 con 0x7f629c06e930 2023-09-29T23:32:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.036+0000 7f62a0ec1700 1 -- 172.21.15.143:0/3814078316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f629c09d5d0 con 0x7f629c06e930 2023-09-29T23:32:18.039 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.036+0000 7f62a0ec1700 1 -- 172.21.15.143:0/3814078316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f629c151050 con 0x7f629c06e930 2023-09-29T23:32:18.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.040+0000 7f62837fe700 1 -- 172.21.15.143:0/3814078316 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f6290013680 con 0x7f629c06e930 2023-09-29T23:32:18.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.040+0000 7f62837fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:18.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.040+0000 7f62837fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:18.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.040+0000 7f62837fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:18.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.040+0000 7f62837fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:18.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.040+0000 7f62837fe700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f628808e830 0x7f6288090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:18.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.040+0000 7f62837fe700 1 -- 172.21.15.143:0/3814078316 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f62900b4ad0 con 0x7f629c06e930 2023-09-29T23:32:18.042 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.040+0000 7f629b7fe700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f628808e830 0x7f6288090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:18.043 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.040+0000 7f629b7fe700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f628808e830 0x7f6288090ce0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f628c0099b0 tx=0x7f628c018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:18.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.048+0000 7f62837fe700 1 -- 172.21.15.143:0/3814078316 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f629c151050 con 0x7f629c06e930 2023-09-29T23:32:18.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.244+0000 7f62a0ec1700 1 -- 172.21.15.143:0/3814078316 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f629c150ff0 con 0x7f629c06e930 2023-09-29T23:32:18.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62837fe700 1 -- 172.21.15.143:0/3814078316 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f6290017100 con 0x7f629c06e930 2023-09-29T23:32:18.248 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:18.248 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:18.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 10 mgrc shutdown 2023-09-29T23:32:18.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 -- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f628808e830 msgr2=0x7f6288090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:18.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f628808e830 0x7f6288090ce0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f628c0099b0 tx=0x7f628c018000 comp rx=0 tx=0).stop 2023-09-29T23:32:18.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 -- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c06e930 msgr2=0x7f629c09c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:18.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c06e930 0x7f629c09c770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f629000ea50 tx=0x7f62900091a0 comp rx=0 tx=0).stop 2023-09-29T23:32:18.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 -- 172.21.15.143:0/3814078316 shutdown_connections 2023-09-29T23:32:18.251 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f628808e830 0x7f6288090ce0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:18.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f629c13c010 0x7f629c09b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:18.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f629c06e930 0x7f629c09c770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:18.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 --2- 172.21.15.143:0/3814078316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f629c066ec0 0x7f629c1500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:18.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 -- 172.21.15.143:0/3814078316 >> 172.21.15.143:0/3814078316 conn(0x7f629c06f200 msgr2=0x7f629c13b320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:18.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 -- 172.21.15.143:0/3814078316 shutdown_connections 2023-09-29T23:32:18.252 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:18.248+0000 7f62a0ec1700 1 -- 172.21.15.143:0/3814078316 wait complete. 2023-09-29T23:32:21.262 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:21.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.372+0000 7f74ad80e700 1 Processor -- start 2023-09-29T23:32:21.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.372+0000 7f74ad80e700 1 -- start start 2023-09-29T23:32:21.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.372+0000 7f74ad80e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f74a806b600 0x7f74a81435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:21.376 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a80a6980 con 0x7f74a8148920 2023-09-29T23:32:21.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a8143f70 con 0x7f74a80a0e70 2023-09-29T23:32:21.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a813ff90 con 0x7f74a806b600 2023-09-29T23:32:21.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad00d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f74a8148920 0x7f74a814ad00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50598/0 (socket says 172.21.15.143:50598) 2023-09-29T23:32:21.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad00d700 1 -- 172.21.15.143:0/1684802775 learned_addr learned my addr 172.21.15.143:0/1684802775 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:32:21.377 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74a7fff700 1 --2- 172.21.15.143:0/1684802775 >> v2:172.21.15.156:3300/0 conn(0x7f74a806b600 0x7f74a81435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:21.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74a77fe700 1 -- 172.21.15.143:0/1684802775 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556617597 0 0) 0x7f74a80a6980 con 0x7f74a8148920 2023-09-29T23:32:21.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74a77fe700 1 -- 172.21.15.143:0/1684802775 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7490003570 con 0x7f74a8148920 2023-09-29T23:32:21.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74a7fff700 1 -- 172.21.15.143:0/1684802775 >> v1:172.21.15.156:6789/0 conn(0x7f74a80a0e70 legacy=0x7f74a806dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74a7fff700 1 -- 172.21.15.143:0/1684802775 >> v1:172.21.15.143:6789/0 conn(0x7f74a8148920 legacy=0x7f74a814ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74a7fff700 1 -- 172.21.15.143:0/1684802775 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8144230 con 0x7f74a806b600 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74a7fff700 1 --2- 172.21.15.143:0/1684802775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74a806b600 0x7f74a81435f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f74980098e0 tx=0x7f74980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8232602669c58e13 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74a77fe700 1 -- 172.21.15.143:0/1684802775 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749800b640 con 0x7f74a806b600 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74a77fe700 1 -- 172.21.15.143:0/1684802775 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749800b7a0 con 0x7f74a806b600 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74a77fe700 1 -- 172.21.15.143:0/1684802775 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749800bb40 con 0x7f74a806b600 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 -- 172.21.15.143:0/1684802775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74a806b600 msgr2=0x7f74a81435f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 --2- 172.21.15.143:0/1684802775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74a806b600 0x7f74a81435f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f74980098e0 tx=0x7f74980091a0 comp rx=0 tx=0).stop 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 -- 172.21.15.143:0/1684802775 shutdown_connections 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 --2- 172.21.15.143:0/1684802775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74a806b600 0x7f74a81435f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 -- 172.21.15.143:0/1684802775 >> 172.21.15.143:0/1684802775 conn(0x7f74a80b4640 msgr2=0x7f74a80b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:21.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 -- 172.21.15.143:0/1684802775 shutdown_connections 2023-09-29T23:32:21.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 -- 172.21.15.143:0/1684802775 wait complete. 2023-09-29T23:32:21.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 Processor -- start 2023-09-29T23:32:21.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.376+0000 7f74ad80e700 1 -- start start 2023-09-29T23:32:21.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ad80e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74a806b600 0x7f74a80a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:21.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ad80e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a80a0e70 0x7f74a813c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:21.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ad80e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74a8148920 0x7f74a813c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:21.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ad80e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a813a090 con 0x7f74a80a0e70 2023-09-29T23:32:21.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ad80e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a8146d00 con 0x7f74a8148920 2023-09-29T23:32:21.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ad80e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a813cd70 con 0x7f74a806b600 2023-09-29T23:32:21.382 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74a7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a80a0e70 0x7f74a813c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ac80c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74a806b600 0x7f74a80a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74a7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a80a0e70 0x7f74a813c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41972/0 (socket says 172.21.15.143:41972) 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ac80c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74a806b600 0x7f74a80a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43364/0 (socket says 172.21.15.143:43364) 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74a7fff700 1 -- 172.21.15.143:0/2898509831 learned_addr learned my addr 172.21.15.143:0/2898509831 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ad00d700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74a8148920 0x7f74a813c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ac80c700 1 -- 172.21.15.143:0/2898509831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74a8148920 msgr2=0x7f74a813c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ac80c700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74a8148920 0x7f74a813c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ac80c700 1 -- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a80a0e70 msgr2=0x7f74a813c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ac80c700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a80a0e70 0x7f74a813c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ac80c700 1 -- 172.21.15.143:0/2898509831 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7498009420 con 0x7f74a806b600 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ac80c700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74a806b600 0x7f74a80a15e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c590 tx=0x7f749c007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74a4ff9700 1 -- 172.21.15.143:0/2898509831 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c011590 con 0x7f74a806b600 2023-09-29T23:32:21.383 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74a4ff9700 1 -- 172.21.15.143:0/2898509831 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c011710 con 0x7f74a806b600 2023-09-29T23:32:21.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ad80e700 1 -- 172.21.15.143:0/2898509831 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a813d030 con 0x7f74a806b600 2023-09-29T23:32:21.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74a4ff9700 1 -- 172.21.15.143:0/2898509831 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c011ad0 con 0x7f74a806b600 2023-09-29T23:32:21.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ad80e700 1 -- 172.21.15.143:0/2898509831 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a80a1b20 con 0x7f74a806b600 2023-09-29T23:32:21.384 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.380+0000 7f74ad80e700 1 -- 172.21.15.143:0/2898509831 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a805ecb0 con 0x7f74a806b600 2023-09-29T23:32:21.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.384+0000 7f74a4ff9700 1 -- 172.21.15.143:0/2898509831 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f749c01b3b0 con 0x7f74a806b600 2023-09-29T23:32:21.385 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.384+0000 7f74a4ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.384+0000 7f74a4ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.384+0000 7f74a4ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.384+0000 7f74a4ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.384+0000 7f74a4ff9700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f749408e830 0x7f7494090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:21.386 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.384+0000 7f74a7fff700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f749408e830 0x7f7494090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:21.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.384+0000 7f74a4ff9700 1 -- 172.21.15.143:0/2898509831 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f749c0b40f0 con 0x7f74a806b600 2023-09-29T23:32:21.387 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.384+0000 7f74a7fff700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f749408e830 0x7f7494090ce0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f74a80b41a0 tx=0x7f7498017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:21.390 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.388+0000 7f74a4ff9700 1 -- 172.21.15.143:0/2898509831 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f749c084ab0 con 0x7f74a806b600 2023-09-29T23:32:21.583 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.580+0000 7f74ad80e700 1 -- 172.21.15.143:0/2898509831 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f74a80a07e0 con 0x7f74a806b600 2023-09-29T23:32:21.584 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.580+0000 7f74a4ff9700 1 -- 172.21.15.143:0/2898509831 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f749c017020 con 0x7f74a806b600 2023-09-29T23:32:21.584 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:21.584 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:21.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.584+0000 7f74ad80e700 10 mgrc shutdown 2023-09-29T23:32:21.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.584+0000 7f74ad80e700 1 -- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f749408e830 msgr2=0x7f7494090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:21.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.584+0000 7f74ad80e700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f749408e830 0x7f7494090ce0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f74a80b41a0 tx=0x7f7498017000 comp rx=0 tx=0).stop 2023-09-29T23:32:21.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.584+0000 7f74ad80e700 1 -- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74a806b600 msgr2=0x7f74a80a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:21.588 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.584+0000 7f74ad80e700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74a806b600 0x7f74a80a15e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c590 tx=0x7f749c007ec0 comp rx=0 tx=0).stop 2023-09-29T23:32:21.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.584+0000 7f74ad80e700 1 -- 172.21.15.143:0/2898509831 shutdown_connections 2023-09-29T23:32:21.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.584+0000 7f74ad80e700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f749408e830 0x7f7494090ce0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:21.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.584+0000 7f74ad80e700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74a8148920 0x7f74a813c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:21.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.584+0000 7f74ad80e700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f74a80a0e70 0x7f74a813c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:21.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.584+0000 7f74ad80e700 1 --2- 172.21.15.143:0/2898509831 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f74a806b600 0x7f74a80a15e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:21.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.584+0000 7f74ad80e700 1 -- 172.21.15.143:0/2898509831 >> 172.21.15.143:0/2898509831 conn(0x7f74a80b4640 msgr2=0x7f74a813f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:21.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.588+0000 7f74ad80e700 1 -- 172.21.15.143:0/2898509831 shutdown_connections 2023-09-29T23:32:21.589 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:21.588+0000 7f74ad80e700 1 -- 172.21.15.143:0/2898509831 wait complete. 2023-09-29T23:32:24.604 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:24.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 Processor -- start 2023-09-29T23:32:24.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 -- start start 2023-09-29T23:32:24.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90300a5690 0x7f90301367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:24.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90300a5ed0 con 0x7f90300a6350 2023-09-29T23:32:24.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90300a6720 con 0x7f90301363f0 2023-09-29T23:32:24.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90300a5c10 con 0x7f90300a5690 2023-09-29T23:32:24.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902ed9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90300a5690 0x7f90301367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:24.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902e59c700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f90300a6350 0x7f903006b360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:50530/0 (socket says 172.21.15.143:50530) 2023-09-29T23:32:24.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902e59c700 1 -- 172.21.15.143:0/2713498329 learned_addr learned my addr 172.21.15.143:0/2713498329 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:32:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902ed9d700 1 -- 172.21.15.143:0/2713498329 >> v1:172.21.15.156:6789/0 conn(0x7f90301363f0 legacy=0x7f90300a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902ed9d700 1 -- 172.21.15.143:0/2713498329 >> v1:172.21.15.143:6789/0 conn(0x7f90300a6350 legacy=0x7f903006b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902ed9d700 1 -- 172.21.15.143:0/2713498329 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903006ba70 con 0x7f90300a5690 2023-09-29T23:32:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902d59a700 1 -- 172.21.15.143:0/2713498329 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3257243485 0 0) 0x7f90300a5ed0 con 0x7f90300a6350 2023-09-29T23:32:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902ed9d700 1 --2- 172.21.15.143:0/2713498329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90300a5690 0x7f90301367c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9020009910 tx=0x7f90200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bbcaeb691f28401 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902d59a700 1 -- 172.21.15.143:0/2713498329 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902000b860 con 0x7f90300a5690 2023-09-29T23:32:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902d59a700 1 -- 172.21.15.143:0/2713498329 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902000b9c0 con 0x7f90300a5690 2023-09-29T23:32:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902d59a700 1 -- 172.21.15.143:0/2713498329 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902000bd60 con 0x7f90300a5690 2023-09-29T23:32:24.720 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 -- 172.21.15.143:0/2713498329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90300a5690 msgr2=0x7f90301367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 --2- 172.21.15.143:0/2713498329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90300a5690 0x7f90301367c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f9020009910 tx=0x7f90200091a0 comp rx=0 tx=0).stop 2023-09-29T23:32:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 -- 172.21.15.143:0/2713498329 shutdown_connections 2023-09-29T23:32:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 --2- 172.21.15.143:0/2713498329 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90300a5690 0x7f90301367c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 -- 172.21.15.143:0/2713498329 >> 172.21.15.143:0/2713498329 conn(0x7f90300b6930 msgr2=0x7f90300b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 -- 172.21.15.143:0/2713498329 shutdown_connections 2023-09-29T23:32:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.716+0000 7f902f59e700 1 -- 172.21.15.143:0/2713498329 wait complete. 2023-09-29T23:32:24.721 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902f59e700 1 Processor -- start 2023-09-29T23:32:24.722 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902f59e700 1 -- start start 2023-09-29T23:32:24.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902f59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90300a5690 0x7f90301326d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902f59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90300a6350 0x7f9030143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902f59e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90301363f0 0x7f903012e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902f59e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903012f920 con 0x7f90300a5690 2023-09-29T23:32:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902f59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903012df60 con 0x7f90300a6350 2023-09-29T23:32:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902f59e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903012fb70 con 0x7f90301363f0 2023-09-29T23:32:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902e59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90300a5690 0x7f90301326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:24.724 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902e59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90300a5690 0x7f90301326d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:60386/0 (socket says 172.21.15.143:60386) 2023-09-29T23:32:24.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902ed9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90301363f0 0x7f903012e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:24.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902e59c700 1 -- 172.21.15.143:0/1869367382 learned_addr learned my addr 172.21.15.143:0/1869367382 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:24.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902dd9b700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90300a6350 0x7f9030143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:24.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902e59c700 1 -- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90301363f0 msgr2=0x7f903012e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:24.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902e59c700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90301363f0 0x7f903012e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:24.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902e59c700 1 -- 172.21.15.143:0/1869367382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90300a6350 msgr2=0x7f9030143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:24.725 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902e59c700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90300a6350 0x7f9030143450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902e59c700 1 -- 172.21.15.143:0/1869367382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9020009420 con 0x7f90300a5690 2023-09-29T23:32:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902e59c700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90300a5690 0x7f90301326d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f902800ccf0 tx=0x7f9028001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f901e7fc700 1 -- 172.21.15.143:0/1869367382 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902800a980 con 0x7f90300a5690 2023-09-29T23:32:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f901e7fc700 1 -- 172.21.15.143:0/1869367382 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902800aae0 con 0x7f90300a5690 2023-09-29T23:32:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902f59e700 1 -- 172.21.15.143:0/1869367382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9030143990 con 0x7f90300a5690 2023-09-29T23:32:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f901e7fc700 1 -- 172.21.15.143:0/1869367382 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902800aea0 con 0x7f90300a5690 2023-09-29T23:32:24.726 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.720+0000 7f902f59e700 1 -- 172.21.15.143:0/1869367382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903012e930 con 0x7f90300a5690 2023-09-29T23:32:24.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.724+0000 7f902f59e700 1 -- 172.21.15.143:0/1869367382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903005ecb0 con 0x7f90300a5690 2023-09-29T23:32:24.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.724+0000 7f901e7fc700 1 -- 172.21.15.143:0/1869367382 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f9028007300 con 0x7f90300a5690 2023-09-29T23:32:24.727 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.724+0000 7f901e7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.728+0000 7f901e7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.728+0000 7f901e7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:24.728 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.728+0000 7f901e7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.728+0000 7f901e7fc700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f901008e8b0 0x7f9010090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.728+0000 7f901e7fc700 1 -- 172.21.15.143:0/1869367382 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f90280b4cd0 con 0x7f90300a5690 2023-09-29T23:32:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.728+0000 7f902dd9b700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f901008e8b0 0x7f9010090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:24.729 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.728+0000 7f902dd9b700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f901008e8b0 0x7f9010090d60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9030131710 tx=0x7f9024009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:24.736 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.732+0000 7f901e7fc700 1 -- 172.21.15.143:0/1869367382 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9028085680 con 0x7f90300a5690 2023-09-29T23:32:24.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 -- 172.21.15.143:0/1869367382 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9030136ae0 con 0x7f90300a5690 2023-09-29T23:32:24.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f901e7fc700 1 -- 172.21.15.143:0/1869367382 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f9028088ca0 con 0x7f90300a5690 2023-09-29T23:32:24.929 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:24.929 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 10 mgrc shutdown 2023-09-29T23:32:24.931 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 -- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f901008e8b0 msgr2=0x7f9010090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f901008e8b0 0x7f9010090d60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9030131710 tx=0x7f9024009040 comp rx=0 tx=0).stop 2023-09-29T23:32:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 -- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90300a5690 msgr2=0x7f90301326d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90300a5690 0x7f90301326d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f902800ccf0 tx=0x7f9028001a80 comp rx=0 tx=0).stop 2023-09-29T23:32:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 -- 172.21.15.143:0/1869367382 shutdown_connections 2023-09-29T23:32:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f901008e8b0 0x7f9010090d60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90301363f0 0x7f903012e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90300a6350 0x7f9030143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 --2- 172.21.15.143:0/1869367382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90300a5690 0x7f90301326d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 -- 172.21.15.143:0/1869367382 >> 172.21.15.143:0/1869367382 conn(0x7f90300b6930 msgr2=0x7f903006c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 -- 172.21.15.143:0/1869367382 shutdown_connections 2023-09-29T23:32:24.932 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:24.928+0000 7f902f59e700 1 -- 172.21.15.143:0/1869367382 wait complete. 2023-09-29T23:32:26.151 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:32:26.153 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:32:27.944 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:28.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.052+0000 7f9ccb590700 1 Processor -- start 2023-09-29T23:32:28.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.052+0000 7f9ccb590700 1 -- start start 2023-09-29T23:32:28.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9cc406efd0 0x7f9cc40a1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:28.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc409fc40 0x7f9cc40a4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:28.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc40a58f0 con 0x7f9cc40a0010 2023-09-29T23:32:28.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc40a05c0 con 0x7f9cc406efd0 2023-09-29T23:32:28.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc40a0880 con 0x7f9cc409fc40 2023-09-29T23:32:28.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cc9d8d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc409fc40 0x7f9cc40a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:28.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cc9d8d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc409fc40 0x7f9cc40a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:48262/0 (socket says 172.21.15.143:48262) 2023-09-29T23:32:28.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cc9d8d700 1 -- 172.21.15.143:0/2028977653 learned_addr learned my addr 172.21.15.143:0/2028977653 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:28.057 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cca58e700 1 --2- 172.21.15.143:0/2028977653 >> v2:172.21.15.156:3300/0 conn(0x7f9cc406efd0 0x7f9cc40a1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:28.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cc9d8d700 1 -- 172.21.15.143:0/2028977653 >> v2:172.21.15.156:3300/0 conn(0x7f9cc406efd0 msgr2=0x7f9cc40a1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:28.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cc9d8d700 1 --2- 172.21.15.143:0/2028977653 >> v2:172.21.15.156:3300/0 conn(0x7f9cc406efd0 0x7f9cc40a1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:28.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cc9d8d700 1 -- 172.21.15.143:0/2028977653 >> v1:172.21.15.143:6789/0 conn(0x7f9cc40a0010 legacy=0x7f9cc40a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:28.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cc9d8d700 1 -- 172.21.15.143:0/2028977653 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc40a1170 con 0x7f9cc409fc40 2023-09-29T23:32:28.058 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cca58e700 1 --2- 172.21.15.143:0/2028977653 >> v2:172.21.15.156:3300/0 conn(0x7f9cc406efd0 0x7f9cc40a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-29T23:32:28.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cc9d8d700 1 --2- 172.21.15.143:0/2028977653 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc409fc40 0x7f9cc40a4260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009f20 tx=0x7f9cbc009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6d5ef5ab3563e93 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:28.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cc958c700 1 -- 172.21.15.143:0/2028977653 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00e070 con 0x7f9cc409fc40 2023-09-29T23:32:28.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cc958c700 1 -- 172.21.15.143:0/2028977653 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc00bb10 con 0x7f9cc409fc40 2023-09-29T23:32:28.059 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9cc958c700 1 -- 172.21.15.143:0/2028977653 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0092e0 con 0x7f9cc409fc40 2023-09-29T23:32:28.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 -- 172.21.15.143:0/2028977653 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc409fc40 msgr2=0x7f9cc40a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:28.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 --2- 172.21.15.143:0/2028977653 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc409fc40 0x7f9cc40a4260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009f20 tx=0x7f9cbc009840 comp rx=0 tx=0).stop 2023-09-29T23:32:28.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 -- 172.21.15.143:0/2028977653 shutdown_connections 2023-09-29T23:32:28.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 --2- 172.21.15.143:0/2028977653 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc409fc40 0x7f9cc40a4260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:28.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 --2- 172.21.15.143:0/2028977653 >> v2:172.21.15.156:3300/0 conn(0x7f9cc406efd0 0x7f9cc40a1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:28.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 -- 172.21.15.143:0/2028977653 >> 172.21.15.143:0/2028977653 conn(0x7f9cc40b2ca0 msgr2=0x7f9cc40b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:28.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 -- 172.21.15.143:0/2028977653 shutdown_connections 2023-09-29T23:32:28.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.056+0000 7f9ccb590700 1 -- 172.21.15.143:0/2028977653 wait complete. 2023-09-29T23:32:28.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9ccb590700 1 Processor -- start 2023-09-29T23:32:28.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9ccb590700 1 -- start start 2023-09-29T23:32:28.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9ccb590700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc406efd0 0x7f9cc40a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:28.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9ccb590700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc409fc40 0x7f9cc40963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:28.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9ccb590700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cc40a0010 0x7f9cc4096910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9ccb590700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc4096fa0 con 0x7f9cc40a0010 2023-09-29T23:32:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9ccb590700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc4097240 con 0x7f9cc409fc40 2023-09-29T23:32:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9ccb590700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc409b860 con 0x7f9cc406efd0 2023-09-29T23:32:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9cca58e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc406efd0 0x7f9cc40a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9ccad8f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cc40a0010 0x7f9cc4096910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9cca58e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc406efd0 0x7f9cc40a6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:48276/0 (socket says 172.21.15.143:48276) 2023-09-29T23:32:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9cca58e700 1 -- 172.21.15.143:0/3674099941 learned_addr learned my addr 172.21.15.143:0/3674099941 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.060+0000 7f9cc9d8d700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc409fc40 0x7f9cc40963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9cca58e700 1 -- 172.21.15.143:0/3674099941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc409fc40 msgr2=0x7f9cc40963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9cca58e700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc409fc40 0x7f9cc40963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:28.065 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9cca58e700 1 -- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cc40a0010 msgr2=0x7f9cc4096910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:28.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9cca58e700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cc40a0010 0x7f9cc4096910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:28.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9cca58e700 1 -- 172.21.15.143:0/3674099941 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc009ae0 con 0x7f9cc406efd0 2023-09-29T23:32:28.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9cca58e700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc406efd0 0x7f9cc40a6890 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00c830 tx=0x7f9cac008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:28.066 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9cc27fc700 1 -- 172.21.15.143:0/3674099941 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac011600 con 0x7f9cc406efd0 2023-09-29T23:32:28.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9cc27fc700 1 -- 172.21.15.143:0/3674099941 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cac011760 con 0x7f9cc406efd0 2023-09-29T23:32:28.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9cc27fc700 1 -- 172.21.15.143:0/3674099941 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cac011b20 con 0x7f9cc406efd0 2023-09-29T23:32:28.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9ccb590700 1 -- 172.21.15.143:0/3674099941 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc40982a0 con 0x7f9cc406efd0 2023-09-29T23:32:28.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9ccb590700 1 -- 172.21.15.143:0/3674099941 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc409a120 con 0x7f9cc406efd0 2023-09-29T23:32:28.067 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.064+0000 7f9ccb590700 1 -- 172.21.15.143:0/3674099941 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc409ba80 con 0x7f9cc406efd0 2023-09-29T23:32:28.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.068+0000 7f9cc27fc700 1 -- 172.21.15.143:0/3674099941 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f9cac018360 con 0x7f9cc406efd0 2023-09-29T23:32:28.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.068+0000 7f9cc27fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.068+0000 7f9cc27fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.068+0000 7f9cc27fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:28.069 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.068+0000 7f9cc27fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:28.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.068+0000 7f9cc27fc700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9ca408e8e0 0x7f9ca4090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:28.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.068+0000 7f9cc9d8d700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9ca408e8e0 0x7f9ca4090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:28.070 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.068+0000 7f9cc27fc700 1 -- 172.21.15.143:0/3674099941 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f9cac0b47e0 con 0x7f9cc406efd0 2023-09-29T23:32:28.071 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.068+0000 7f9cc9d8d700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9ca408e8e0 0x7f9ca4090d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009880 tx=0x7f9cbc018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:28.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.072+0000 7f9cc27fc700 1 -- 172.21.15.143:0/3674099941 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9cac085190 con 0x7f9cc406efd0 2023-09-29T23:32:28.274 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.272+0000 7f9ccb590700 1 -- 172.21.15.143:0/3674099941 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f9cc405ecb0 con 0x7f9cc406efd0 2023-09-29T23:32:28.275 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.272+0000 7f9cc27fc700 1 -- 172.21.15.143:0/3674099941 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f9cac0887b0 con 0x7f9cc406efd0 2023-09-29T23:32:28.275 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:28.275 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:28.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 10 mgrc shutdown 2023-09-29T23:32:28.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 -- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9ca408e8e0 msgr2=0x7f9ca4090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:28.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9ca408e8e0 0x7f9ca4090d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc009880 tx=0x7f9cbc018000 comp rx=0 tx=0).stop 2023-09-29T23:32:28.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 -- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc406efd0 msgr2=0x7f9cc40a6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:28.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc406efd0 0x7f9cc40a6890 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9cac00c830 tx=0x7f9cac008040 comp rx=0 tx=0).stop 2023-09-29T23:32:28.279 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 -- 172.21.15.143:0/3674099941 shutdown_connections 2023-09-29T23:32:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f9ca408e8e0 0x7f9ca4090d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9cc40a0010 0x7f9cc4096910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc409fc40 0x7f9cc40963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 --2- 172.21.15.143:0/3674099941 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f9cc406efd0 0x7f9cc40a6890 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 -- 172.21.15.143:0/3674099941 >> 172.21.15.143:0/3674099941 conn(0x7f9cc40b2ca0 msgr2=0x7f9cc406bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 -- 172.21.15.143:0/3674099941 shutdown_connections 2023-09-29T23:32:28.280 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:28.276+0000 7f9ccb590700 1 -- 172.21.15.143:0/3674099941 wait complete. 2023-09-29T23:32:31.294 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:31.405 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7d5979700 1 Processor -- start 2023-09-29T23:32:31.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7d5979700 1 -- start start 2023-09-29T23:32:31.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7d5979700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd7d006f060 0x7fd7d00a3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7d5979700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a1ab0 0x7fd7d009fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7d5979700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd7d00a2190 0x7fd7d00a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7d5979700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d00a3e50 con 0x7fd7d006f060 2023-09-29T23:32:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7d5979700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d006f430 con 0x7fd7d00a2190 2023-09-29T23:32:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7d5979700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00a4060 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7cffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a1ab0 0x7fd7d009fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:31.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7d4977700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd7d006f060 0x7fd7d00a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:31.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7cffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a1ab0 0x7fd7d009fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:48290/0 (socket says 172.21.15.143:48290) 2023-09-29T23:32:31.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7cffff700 1 -- 172.21.15.143:0/2186168881 learned_addr learned my addr 172.21.15.143:0/2186168881 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:31.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.404+0000 7fd7d5178700 1 --2- 172.21.15.143:0/2186168881 >> v2:172.21.15.156:3300/0 conn(0x7fd7d00a2190 0x7fd7d00a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d4977700 1 -- 172.21.15.143:0/2186168881 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a1ab0 msgr2=0x7fd7d009fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d4977700 1 --2- 172.21.15.143:0/2186168881 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a1ab0 0x7fd7d009fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d4977700 1 -- 172.21.15.143:0/2186168881 >> v2:172.21.15.156:3300/0 conn(0x7fd7d00a2190 msgr2=0x7fd7d00a02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d4977700 1 --2- 172.21.15.143:0/2186168881 >> v2:172.21.15.156:3300/0 conn(0x7fd7d00a2190 0x7fd7d00a02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d4977700 1 -- 172.21.15.143:0/2186168881 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d00a4320 con 0x7fd7d006f060 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d4977700 1 --2- 172.21.15.143:0/2186168881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d006f060 0x7fd7d00a3910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd7d009bf00 tx=0x7fd7c400d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ae1e3231c94e83 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7cf7fe700 1 -- 172.21.15.143:0/2186168881 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4012070 con 0x7fd7d006f060 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7cf7fe700 1 -- 172.21.15.143:0/2186168881 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c400d2e0 con 0x7fd7d006f060 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7cf7fe700 1 -- 172.21.15.143:0/2186168881 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c400f900 con 0x7fd7d006f060 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d5979700 1 -- 172.21.15.143:0/2186168881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d006f060 msgr2=0x7fd7d00a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d5979700 1 --2- 172.21.15.143:0/2186168881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d006f060 0x7fd7d00a3910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd7d009bf00 tx=0x7fd7c400d630 comp rx=0 tx=0).stop 2023-09-29T23:32:31.411 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d5979700 1 -- 172.21.15.143:0/2186168881 shutdown_connections 2023-09-29T23:32:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d5979700 1 --2- 172.21.15.143:0/2186168881 >> v2:172.21.15.156:3300/0 conn(0x7fd7d00a2190 0x7fd7d00a02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d5979700 1 --2- 172.21.15.143:0/2186168881 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a1ab0 0x7fd7d009fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d5979700 1 --2- 172.21.15.143:0/2186168881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d006f060 0x7fd7d00a3910 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d5979700 1 -- 172.21.15.143:0/2186168881 >> 172.21.15.143:0/2186168881 conn(0x7fd7d006a470 msgr2=0x7fd7d006a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d5979700 1 -- 172.21.15.143:0/2186168881 shutdown_connections 2023-09-29T23:32:31.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.408+0000 7fd7d5979700 1 -- 172.21.15.143:0/2186168881 wait complete. 2023-09-29T23:32:31.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5979700 1 Processor -- start 2023-09-29T23:32:31.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5979700 1 -- start start 2023-09-29T23:32:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5979700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d006f060 0x7fd7d0099f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5979700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00a1ab0 0x7fd7d0099310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5979700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a2190 0x7fd7d00977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5979700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d009a900 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5979700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d009ab90 con 0x7fd7d006f060 2023-09-29T23:32:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5979700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0095b20 con 0x7fd7d00a2190 2023-09-29T23:32:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5178700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a2190 0x7fd7d00977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7cffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00a1ab0 0x7fd7d0099310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5178700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a2190 0x7fd7d00977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:48304/0 (socket says 172.21.15.143:48304) 2023-09-29T23:32:31.415 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5178700 1 -- 172.21.15.143:0/358905346 learned_addr learned my addr 172.21.15.143:0/358905346 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:31.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d4977700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d006f060 0x7fd7d0099f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:31.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7cffff700 1 -- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a2190 msgr2=0x7fd7d00977e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:31.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7cffff700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a2190 0x7fd7d00977e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:31.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7cffff700 1 -- 172.21.15.143:0/358905346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d006f060 msgr2=0x7fd7d0099f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:31.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7cffff700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d006f060 0x7fd7d0099f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:31.416 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7cffff700 1 -- 172.21.15.143:0/358905346 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c400d8d0 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7cffff700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00a1ab0 0x7fd7d0099310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0006d40 tx=0x7fd7c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:31.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7ccff9700 1 -- 172.21.15.143:0/358905346 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0013680 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7ccff9700 1 -- 172.21.15.143:0/358905346 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c00137e0 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5979700 1 -- 172.21.15.143:0/358905346 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d009a4b0 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7ccff9700 1 -- 172.21.15.143:0/358905346 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0013b60 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.412+0000 7fd7d5979700 1 -- 172.21.15.143:0/358905346 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d00999f0 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.416+0000 7fd7d5979700 1 -- 172.21.15.143:0/358905346 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d005ecb0 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.416+0000 7fd7ccff9700 1 -- 172.21.15.143:0/358905346 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fd7c000c3e0 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.416+0000 7fd7ccff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:31.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.420+0000 7fd7ccff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:31.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.420+0000 7fd7ccff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:31.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.420+0000 7fd7ccff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:31.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.420+0000 7fd7ccff9700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7bc08e8b0 0x7fd7bc090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:31.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.420+0000 7fd7ccff9700 1 -- 172.21.15.143:0/358905346 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fd7c00b5050 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.420+0000 7fd7d4977700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7bc08e8b0 0x7fd7bc090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:31.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.420+0000 7fd7d4977700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7bc08e8b0 0x7fd7bc090d60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400d670 tx=0x7fd7c401c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:31.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.424+0000 7fd7ccff9700 1 -- 172.21.15.143:0/358905346 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd7c0085aa0 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.617 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.616+0000 7fd7d5979700 1 -- 172.21.15.143:0/358905346 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd7d009dd50 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.618 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.616+0000 7fd7ccff9700 1 -- 172.21.15.143:0/358905346 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fd7c0017100 con 0x7fd7d00a1ab0 2023-09-29T23:32:31.618 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:31.618 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 10 mgrc shutdown 2023-09-29T23:32:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 -- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7bc08e8b0 msgr2=0x7fd7bc090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7bc08e8b0 0x7fd7bc090d60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400d670 tx=0x7fd7c401c000 comp rx=0 tx=0).stop 2023-09-29T23:32:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 -- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00a1ab0 msgr2=0x7fd7d0099310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00a1ab0 0x7fd7d0099310 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0006d40 tx=0x7fd7c0009250 comp rx=0 tx=0).stop 2023-09-29T23:32:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 -- 172.21.15.143:0/358905346 shutdown_connections 2023-09-29T23:32:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd7bc08e8b0 0x7fd7bc090d60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd7d00a2190 0x7fd7d00977e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd7d00a1ab0 0x7fd7d0099310 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:31.623 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 --2- 172.21.15.143:0/358905346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7d006f060 0x7fd7d0099f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:31.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 -- 172.21.15.143:0/358905346 >> 172.21.15.143:0/358905346 conn(0x7fd7d006a470 msgr2=0x7fd7d00683c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:31.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 -- 172.21.15.143:0/358905346 shutdown_connections 2023-09-29T23:32:31.624 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:31.620+0000 7fd7d5979700 1 -- 172.21.15.143:0/358905346 wait complete. 2023-09-29T23:32:34.636 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:34.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd368668700 1 Processor -- start 2023-09-29T23:32:34.750 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd368668700 1 -- start start 2023-09-29T23:32:34.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd368668700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd360132bc0 0x7fd3600a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:34.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd368668700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b32e0 0x7fd36006b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:34.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd368668700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36006c3c0 con 0x7fd3600b2f10 2023-09-29T23:32:34.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd368668700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36006ad00 con 0x7fd360132bc0 2023-09-29T23:32:34.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd368668700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36006ae40 con 0x7fd3600b32e0 2023-09-29T23:32:34.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd366c05700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b32e0 0x7fd36006b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:34.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd366c05700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b32e0 0x7fd36006b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:58034/0 (socket says 172.21.15.143:58034) 2023-09-29T23:32:34.752 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd366c05700 1 -- 172.21.15.143:0/458199076 learned_addr learned my addr 172.21.15.143:0/458199076 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:34.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd366404700 1 --2- 172.21.15.143:0/458199076 >> v2:172.21.15.156:3300/0 conn(0x7fd360132bc0 0x7fd3600a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:34.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd366c05700 1 -- 172.21.15.143:0/458199076 >> v2:172.21.15.156:3300/0 conn(0x7fd360132bc0 msgr2=0x7fd3600a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:34.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd366c05700 1 --2- 172.21.15.143:0/458199076 >> v2:172.21.15.156:3300/0 conn(0x7fd360132bc0 0x7fd3600a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:34.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd366c05700 1 -- 172.21.15.143:0/458199076 >> v1:172.21.15.156:6789/0 conn(0x7fd3600b2f10 legacy=0x7fd36006c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:34.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd366c05700 1 -- 172.21.15.143:0/458199076 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36006cd90 con 0x7fd3600b32e0 2023-09-29T23:32:34.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd366c05700 1 --2- 172.21.15.143:0/458199076 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b32e0 0x7fd36006b730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd3580099b0 tx=0x7fd358009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6248d36dad7c584 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:34.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.748+0000 7fd365402700 1 -- 172.21.15.143:0/458199076 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800b860 con 0x7fd3600b32e0 2023-09-29T23:32:34.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd365402700 1 -- 172.21.15.143:0/458199076 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35800b9c0 con 0x7fd3600b32e0 2023-09-29T23:32:34.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd365402700 1 -- 172.21.15.143:0/458199076 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800bd60 con 0x7fd3600b32e0 2023-09-29T23:32:34.754 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 -- 172.21.15.143:0/458199076 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b32e0 msgr2=0x7fd36006b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 --2- 172.21.15.143:0/458199076 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b32e0 0x7fd36006b730 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd3580099b0 tx=0x7fd358009250 comp rx=0 tx=0).stop 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 -- 172.21.15.143:0/458199076 shutdown_connections 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 --2- 172.21.15.143:0/458199076 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b32e0 0x7fd36006b730 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 --2- 172.21.15.143:0/458199076 >> v2:172.21.15.156:3300/0 conn(0x7fd360132bc0 0x7fd3600a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 -- 172.21.15.143:0/458199076 >> 172.21.15.143:0/458199076 conn(0x7fd360067dc0 msgr2=0x7fd3600b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 -- 172.21.15.143:0/458199076 shutdown_connections 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 -- 172.21.15.143:0/458199076 wait complete. 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 Processor -- start 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 -- start start 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b2f10 0x7fd3600a8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3600b32e0 0x7fd36012b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:34.755 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd360132bc0 0x7fd36012bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:34.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600a7de0 con 0x7fd3600b32e0 2023-09-29T23:32:34.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600a75a0 con 0x7fd360132bc0 2023-09-29T23:32:34.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3600a89c0 con 0x7fd3600b2f10 2023-09-29T23:32:34.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd366404700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b2f10 0x7fd3600a8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:34.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd366404700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b2f10 0x7fd3600a8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:58042/0 (socket says 172.21.15.143:58042) 2023-09-29T23:32:34.756 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd366404700 1 -- 172.21.15.143:0/800069367 learned_addr learned my addr 172.21.15.143:0/800069367 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd365c03700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3600b32e0 0x7fd36012b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd366c05700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd360132bc0 0x7fd36012bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd366404700 1 -- 172.21.15.143:0/800069367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd360132bc0 msgr2=0x7fd36012bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:34.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd366404700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd360132bc0 0x7fd36012bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:34.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd366404700 1 -- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3600b32e0 msgr2=0x7fd36012b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:34.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd366404700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3600b32e0 0x7fd36012b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:34.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd366404700 1 -- 172.21.15.143:0/800069367 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580094f0 con 0x7fd3600b2f10 2023-09-29T23:32:34.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd366404700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b2f10 0x7fd3600a8480 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd35000d860 tx=0x7fd350008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:34.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd3567fc700 1 -- 172.21.15.143:0/800069367 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3500125f0 con 0x7fd3600b2f10 2023-09-29T23:32:34.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd3567fc700 1 -- 172.21.15.143:0/800069367 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd350012750 con 0x7fd3600b2f10 2023-09-29T23:32:34.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.752+0000 7fd368668700 1 -- 172.21.15.143:0/800069367 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd360145760 con 0x7fd3600b2f10 2023-09-29T23:32:34.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.756+0000 7fd3567fc700 1 -- 172.21.15.143:0/800069367 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd350012b10 con 0x7fd3600b2f10 2023-09-29T23:32:34.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.756+0000 7fd368668700 1 -- 172.21.15.143:0/800069367 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36012c2e0 con 0x7fd3600b2f10 2023-09-29T23:32:34.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.756+0000 7fd368668700 1 -- 172.21.15.143:0/800069367 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd360064860 con 0x7fd3600b2f10 2023-09-29T23:32:34.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.760+0000 7fd3567fc700 1 -- 172.21.15.143:0/800069367 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fd35001e460 con 0x7fd3600b2f10 2023-09-29T23:32:34.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.760+0000 7fd3567fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:34.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.760+0000 7fd3567fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:34.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.760+0000 7fd3567fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:34.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.760+0000 7fd3567fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:34.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.760+0000 7fd3567fc700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd34408e8b0 0x7fd344090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:34.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.760+0000 7fd365c03700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd34408e8b0 0x7fd344090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:34.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.760+0000 7fd3567fc700 1 -- 172.21.15.143:0/800069367 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fd3500b5790 con 0x7fd3600b2f10 2023-09-29T23:32:34.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.760+0000 7fd365c03700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd34408e8b0 0x7fd344090d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd3601437b0 tx=0x7fd35c0016f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:34.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.764+0000 7fd3567fc700 1 -- 172.21.15.143:0/800069367 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd350086150 con 0x7fd3600b2f10 2023-09-29T23:32:34.959 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.956+0000 7fd368668700 1 -- 172.21.15.143:0/800069367 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd360064860 con 0x7fd3600b2f10 2023-09-29T23:32:34.960 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd3567fc700 1 -- 172.21.15.143:0/800069367 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fd360064860 con 0x7fd3600b2f10 2023-09-29T23:32:34.960 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:34.960 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:34.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 10 mgrc shutdown 2023-09-29T23:32:34.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 -- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd34408e8b0 msgr2=0x7fd344090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:34.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd34408e8b0 0x7fd344090d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd3601437b0 tx=0x7fd35c0016f0 comp rx=0 tx=0).stop 2023-09-29T23:32:34.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 -- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b2f10 msgr2=0x7fd3600a8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:34.963 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b2f10 0x7fd3600a8480 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd35000d860 tx=0x7fd350008040 comp rx=0 tx=0).stop 2023-09-29T23:32:34.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 -- 172.21.15.143:0/800069367 shutdown_connections 2023-09-29T23:32:34.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd34408e8b0 0x7fd344090d60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:34.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd360132bc0 0x7fd36012bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:34.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd3600b32e0 0x7fd36012b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:34.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 --2- 172.21.15.143:0/800069367 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd3600b2f10 0x7fd3600a8480 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:34.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 -- 172.21.15.143:0/800069367 >> 172.21.15.143:0/800069367 conn(0x7fd360067dc0 msgr2=0x7fd3600b0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:34.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 -- 172.21.15.143:0/800069367 shutdown_connections 2023-09-29T23:32:34.964 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:34.960+0000 7fd368668700 1 -- 172.21.15.143:0/800069367 wait complete. 2023-09-29T23:32:37.976 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:38.087 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.084+0000 7f2723b7a700 1 Processor -- start 2023-09-29T23:32:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.084+0000 7f2723b7a700 1 -- start start 2023-09-29T23:32:38.088 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c06ee30 0x7f271c0a4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f271c0a5ef0 0x7f271c099e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f271c0b3c60 con 0x7f271c06f200 2023-09-29T23:32:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271c13ffe0 con 0x7f271c0a5ef0 2023-09-29T23:32:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c09a7e0 con 0x7f271c06ee30 2023-09-29T23:32:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2721916700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c06ee30 0x7f271c0a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2721916700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c06ee30 0x7f271c0a4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:58050/0 (socket says 172.21.15.143:58050) 2023-09-29T23:32:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2721916700 1 -- 172.21.15.143:0/537331021 learned_addr learned my addr 172.21.15.143:0/537331021 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:38.089 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2721115700 1 --2- 172.21.15.143:0/537331021 >> v2:172.21.15.156:3300/0 conn(0x7f271c0a5ef0 0x7f271c099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:38.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2721916700 1 -- 172.21.15.143:0/537331021 >> v2:172.21.15.156:3300/0 conn(0x7f271c0a5ef0 msgr2=0x7f271c099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:38.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2721916700 1 --2- 172.21.15.143:0/537331021 >> v2:172.21.15.156:3300/0 conn(0x7f271c0a5ef0 0x7f271c099e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:38.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2721916700 1 -- 172.21.15.143:0/537331021 >> v1:172.21.15.156:6789/0 conn(0x7f271c06f200 legacy=0x7f271c13f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:38.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2721916700 1 -- 172.21.15.143:0/537331021 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c09aa80 con 0x7f271c06ee30 2023-09-29T23:32:38.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2721916700 1 --2- 172.21.15.143:0/537331021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c06ee30 0x7f271c0a4610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f270c0099b0 tx=0x7f270c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5037369370e49a04 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:38.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2720914700 1 -- 172.21.15.143:0/537331021 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c009ea0 con 0x7f271c06ee30 2023-09-29T23:32:38.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2720914700 1 -- 172.21.15.143:0/537331021 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f270c00b860 con 0x7f271c06ee30 2023-09-29T23:32:38.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2720914700 1 -- 172.21.15.143:0/537331021 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f270c00bc00 con 0x7f271c06ee30 2023-09-29T23:32:38.091 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 -- 172.21.15.143:0/537331021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c06ee30 msgr2=0x7f271c0a4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:38.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 --2- 172.21.15.143:0/537331021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c06ee30 0x7f271c0a4610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f270c0099b0 tx=0x7f270c009250 comp rx=0 tx=0).stop 2023-09-29T23:32:38.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 -- 172.21.15.143:0/537331021 shutdown_connections 2023-09-29T23:32:38.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 --2- 172.21.15.143:0/537331021 >> v2:172.21.15.156:3300/0 conn(0x7f271c0a5ef0 0x7f271c099e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:38.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 --2- 172.21.15.143:0/537331021 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c06ee30 0x7f271c0a4610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:38.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 -- 172.21.15.143:0/537331021 >> 172.21.15.143:0/537331021 conn(0x7f271c067720 msgr2=0x7f271c0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:38.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 -- 172.21.15.143:0/537331021 shutdown_connections 2023-09-29T23:32:38.092 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.088+0000 7f2723b7a700 1 -- 172.21.15.143:0/537331021 wait complete. 2023-09-29T23:32:38.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2723b7a700 1 Processor -- start 2023-09-29T23:32:38.093 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2723b7a700 1 -- start start 2023-09-29T23:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2723b7a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f271c06ee30 0x7f271c150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2723b7a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f271c06f200 0x7f271c150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2723b7a700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c0a5ef0 0x7f271c09e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2723b7a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c151310 con 0x7f271c06ee30 2023-09-29T23:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2723b7a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271c06c9e0 con 0x7f271c06f200 2023-09-29T23:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2723b7a700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f271c1503e0 con 0x7f271c0a5ef0 2023-09-29T23:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2722117700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c0a5ef0 0x7f271c09e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2722117700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c0a5ef0 0x7f271c09e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:58054/0 (socket says 172.21.15.143:58054) 2023-09-29T23:32:38.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2722117700 1 -- 172.21.15.143:0/961306603 learned_addr learned my addr 172.21.15.143:0/961306603 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2721916700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f271c06ee30 0x7f271c150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2721115700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f271c06f200 0x7f271c150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2722117700 1 -- 172.21.15.143:0/961306603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f271c06f200 msgr2=0x7f271c150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2722117700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f271c06f200 0x7f271c150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2722117700 1 -- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f271c06ee30 msgr2=0x7f271c150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2722117700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f271c06ee30 0x7f271c150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2722117700 1 -- 172.21.15.143:0/961306603 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f270c0094f0 con 0x7f271c0a5ef0 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2721115700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f271c06f200 0x7f271c150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2721916700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f271c06ee30 0x7f271c150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2722117700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c0a5ef0 0x7f271c09e6f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f271400c910 tx=0x7f27140016f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2711ffb700 1 -- 172.21.15.143:0/961306603 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271400a7e0 con 0x7f271c0a5ef0 2023-09-29T23:32:38.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2711ffb700 1 -- 172.21.15.143:0/961306603 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271400a940 con 0x7f271c0a5ef0 2023-09-29T23:32:38.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2711ffb700 1 -- 172.21.15.143:0/961306603 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271400ad00 con 0x7f271c0a5ef0 2023-09-29T23:32:38.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2723b7a700 1 -- 172.21.15.143:0/961306603 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271c151520 con 0x7f271c0a5ef0 2023-09-29T23:32:38.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.092+0000 7f2723b7a700 1 -- 172.21.15.143:0/961306603 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271c06e970 con 0x7f271c0a5ef0 2023-09-29T23:32:38.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.096+0000 7f2723b7a700 1 -- 172.21.15.143:0/961306603 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271c0a1360 con 0x7f271c0a5ef0 2023-09-29T23:32:38.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.096+0000 7f2711ffb700 1 -- 172.21.15.143:0/961306603 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f2714012630 con 0x7f271c0a5ef0 2023-09-29T23:32:38.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.096+0000 7f2711ffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:38.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.096+0000 7f2711ffb700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:38.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.096+0000 7f2711ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:38.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.096+0000 7f2711ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:38.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.096+0000 7f2711ffb700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f270008e8b0 0x7f2700090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:38.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.096+0000 7f2721916700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f270008e8b0 0x7f2700090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:38.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.096+0000 7f2711ffb700 1 -- 172.21.15.143:0/961306603 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f27140b4ac0 con 0x7f271c0a5ef0 2023-09-29T23:32:38.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.096+0000 7f2721916700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f270008e8b0 0x7f2700090d60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f270c0099b0 tx=0x7f270c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:38.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.104+0000 7f2711ffb700 1 -- 172.21.15.143:0/961306603 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f271c0a1360 con 0x7f271c0a5ef0 2023-09-29T23:32:38.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 -- 172.21.15.143:0/961306603 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f271c0a1360 con 0x7f271c0a5ef0 2023-09-29T23:32:38.297 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2711ffb700 1 -- 172.21.15.143:0/961306603 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f271c0a1360 con 0x7f271c0a5ef0 2023-09-29T23:32:38.297 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:38.297 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:38.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 10 mgrc shutdown 2023-09-29T23:32:38.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 -- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f270008e8b0 msgr2=0x7f2700090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:38.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f270008e8b0 0x7f2700090d60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f270c0099b0 tx=0x7f270c017000 comp rx=0 tx=0).stop 2023-09-29T23:32:38.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 -- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c0a5ef0 msgr2=0x7f271c09e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:38.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c0a5ef0 0x7f271c09e6f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f271400c910 tx=0x7f27140016f0 comp rx=0 tx=0).stop 2023-09-29T23:32:38.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 -- 172.21.15.143:0/961306603 shutdown_connections 2023-09-29T23:32:38.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f270008e8b0 0x7f2700090d60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:38.301 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f271c0a5ef0 0x7f271c09e6f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:38.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f271c06f200 0x7f271c150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:38.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 --2- 172.21.15.143:0/961306603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f271c06ee30 0x7f271c150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:38.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 -- 172.21.15.143:0/961306603 >> 172.21.15.143:0/961306603 conn(0x7f271c067720 msgr2=0x7f271c0a0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:38.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 -- 172.21.15.143:0/961306603 shutdown_connections 2023-09-29T23:32:38.302 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:38.296+0000 7f2723b7a700 1 -- 172.21.15.143:0/961306603 wait complete. 2023-09-29T23:32:41.312 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:41.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1768d76700 1 Processor -- start 2023-09-29T23:32:41.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1768d76700 1 -- start start 2023-09-29T23:32:41.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1768d76700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f17640a6350 0x7f176406b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:41.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1768d76700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f17640a4f80 0x7f17640a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:41.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1768d76700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17640a5ed0 con 0x7f176406b8a0 2023-09-29T23:32:41.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1768d76700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17640a6720 con 0x7f17640a4f80 2023-09-29T23:32:41.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1768d76700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17640a5960 con 0x7f17640a6350 2023-09-29T23:32:41.428 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1762ffd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f176406b8a0 0x7f17641363f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:57522/0 (socket says 172.21.15.143:57522) 2023-09-29T23:32:41.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1762ffd700 1 -- 172.21.15.143:0/410826357 learned_addr learned my addr 172.21.15.143:0/410826357 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:32:41.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1763fff700 1 --2- 172.21.15.143:0/410826357 >> v2:172.21.15.143:3300/0 conn(0x7f17640a4f80 0x7f17640a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:41.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f17637fe700 1 --2- 172.21.15.143:0/410826357 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f17640a6350 0x7f176406b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:41.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f17627fc700 1 -- 172.21.15.143:0/410826357 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699513528 0 0) 0x7f17640a5ed0 con 0x7f176406b8a0 2023-09-29T23:32:41.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f17627fc700 1 -- 172.21.15.143:0/410826357 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f174c003570 con 0x7f176406b8a0 2023-09-29T23:32:41.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1763fff700 1 -- 172.21.15.143:0/410826357 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f17640a6350 msgr2=0x7f176406b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:41.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1763fff700 1 --2- 172.21.15.143:0/410826357 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f17640a6350 0x7f176406b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:41.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1763fff700 1 -- 172.21.15.143:0/410826357 >> v1:172.21.15.143:6789/0 conn(0x7f176406b8a0 legacy=0x7f17641363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:41.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f1763fff700 1 -- 172.21.15.143:0/410826357 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17640a5c10 con 0x7f17640a4f80 2023-09-29T23:32:41.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.424+0000 7f17637fe700 1 --2- 172.21.15.143:0/410826357 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f17640a6350 0x7f176406b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:41.429 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1763fff700 1 --2- 172.21.15.143:0/410826357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17640a4f80 0x7f17640a5420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f175c009e60 tx=0x7f175c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7002c329caaca348 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:41.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17627fc700 1 -- 172.21.15.143:0/410826357 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c00e070 con 0x7f17640a4f80 2023-09-29T23:32:41.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17627fc700 1 -- 172.21.15.143:0/410826357 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175c00bab0 con 0x7f17640a4f80 2023-09-29T23:32:41.430 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17627fc700 1 -- 172.21.15.143:0/410826357 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175c00be70 con 0x7f17640a4f80 2023-09-29T23:32:41.432 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 -- 172.21.15.143:0/410826357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17640a4f80 msgr2=0x7f17640a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 --2- 172.21.15.143:0/410826357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17640a4f80 0x7f17640a5420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f175c009e60 tx=0x7f175c009780 comp rx=0 tx=0).stop 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 -- 172.21.15.143:0/410826357 shutdown_connections 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 --2- 172.21.15.143:0/410826357 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17640a4f80 0x7f17640a5420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 --2- 172.21.15.143:0/410826357 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f17640a6350 0x7f176406b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 -- 172.21.15.143:0/410826357 >> 172.21.15.143:0/410826357 conn(0x7f17640b6930 msgr2=0x7f17640b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 -- 172.21.15.143:0/410826357 shutdown_connections 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 -- 172.21.15.143:0/410826357 wait complete. 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 Processor -- start 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 -- start start 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f176406b8a0 0x7f1764132100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17640a4f80 0x7f17640b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:41.433 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17640a6350 0x7f17640b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17640b4330 con 0x7f17640a6350 2023-09-29T23:32:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17640b5d30 con 0x7f17640a4f80 2023-09-29T23:32:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1768d76700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17640b5f40 con 0x7f176406b8a0 2023-09-29T23:32:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17637fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f176406b8a0 0x7f1764132100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17637fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f176406b8a0 0x7f1764132100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:58082/0 (socket says 172.21.15.143:58082) 2023-09-29T23:32:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17637fe700 1 -- 172.21.15.143:0/625225959 learned_addr learned my addr 172.21.15.143:0/625225959 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1763fff700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17640a6350 0x7f17640b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1762ffd700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17640a4f80 0x7f17640b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17637fe700 1 -- 172.21.15.143:0/625225959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17640a4f80 msgr2=0x7f17640b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17637fe700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17640a4f80 0x7f17640b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:41.434 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17637fe700 1 -- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17640a6350 msgr2=0x7f17640b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:41.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17637fe700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17640a6350 0x7f17640b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:41.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17637fe700 1 -- 172.21.15.143:0/625225959 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1758009420 con 0x7f176406b8a0 2023-09-29T23:32:41.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f1763fff700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17640a6350 0x7f17640b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:32:41.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.428+0000 7f17637fe700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f176406b8a0 0x7f1764132100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f175800ecb0 tx=0x7f17580091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:41.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.432+0000 7f174b7fe700 1 -- 172.21.15.143:0/625225959 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175800c9d0 con 0x7f176406b8a0 2023-09-29T23:32:41.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.432+0000 7f174b7fe700 1 -- 172.21.15.143:0/625225959 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175800cb30 con 0x7f176406b8a0 2023-09-29T23:32:41.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.432+0000 7f1768d76700 1 -- 172.21.15.143:0/625225959 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175c009a20 con 0x7f176406b8a0 2023-09-29T23:32:41.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.432+0000 7f174b7fe700 1 -- 172.21.15.143:0/625225959 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175800ceb0 con 0x7f176406b8a0 2023-09-29T23:32:41.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.432+0000 7f1768d76700 1 -- 172.21.15.143:0/625225959 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176412e880 con 0x7f176406b8a0 2023-09-29T23:32:41.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.432+0000 7f1768d76700 1 -- 172.21.15.143:0/625225959 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17640b4fa0 con 0x7f176406b8a0 2023-09-29T23:32:41.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.436+0000 7f174b7fe700 1 -- 172.21.15.143:0/625225959 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f1758013680 con 0x7f176406b8a0 2023-09-29T23:32:41.436 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.436+0000 7f174b7fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:41.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.436+0000 7f174b7fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:41.437 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.436+0000 7f174b7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:41.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.436+0000 7f174b7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:41.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.436+0000 7f174b7fe700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f175008e8b0 0x7f1750090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:41.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.436+0000 7f1762ffd700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f175008e8b0 0x7f1750090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:41.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.436+0000 7f174b7fe700 1 -- 172.21.15.143:0/625225959 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f17580b4c00 con 0x7f176406b8a0 2023-09-29T23:32:41.438 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.436+0000 7f1762ffd700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f175008e8b0 0x7f1750090d60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f17640b14c0 tx=0x7f1754009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:41.444 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.444+0000 7f174b7fe700 1 -- 172.21.15.143:0/625225959 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f17580855b0 con 0x7f176406b8a0 2023-09-29T23:32:41.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.632+0000 7f1768d76700 1 -- 172.21.15.143:0/625225959 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f176405ecb0 con 0x7f176406b8a0 2023-09-29T23:32:41.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.632+0000 7f174b7fe700 1 -- 172.21.15.143:0/625225959 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f1758017100 con 0x7f176406b8a0 2023-09-29T23:32:41.634 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:41.634 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:41.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.632+0000 7f1768d76700 10 mgrc shutdown 2023-09-29T23:32:41.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.632+0000 7f1768d76700 1 -- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f175008e8b0 msgr2=0x7f1750090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:41.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.632+0000 7f1768d76700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f175008e8b0 0x7f1750090d60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f17640b14c0 tx=0x7f1754009040 comp rx=0 tx=0).stop 2023-09-29T23:32:41.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.632+0000 7f1768d76700 1 -- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f176406b8a0 msgr2=0x7f1764132100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:41.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.632+0000 7f1768d76700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f176406b8a0 0x7f1764132100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f175800ecb0 tx=0x7f17580091a0 comp rx=0 tx=0).stop 2023-09-29T23:32:41.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.636+0000 7f1768d76700 1 -- 172.21.15.143:0/625225959 shutdown_connections 2023-09-29T23:32:41.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.636+0000 7f1768d76700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f175008e8b0 0x7f1750090d60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:41.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.636+0000 7f1768d76700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f17640a6350 0x7f17640b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:41.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.636+0000 7f1768d76700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f17640a4f80 0x7f17640b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:41.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.636+0000 7f1768d76700 1 --2- 172.21.15.143:0/625225959 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f176406b8a0 0x7f1764132100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:41.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.636+0000 7f1768d76700 1 -- 172.21.15.143:0/625225959 >> 172.21.15.143:0/625225959 conn(0x7f17640b6930 msgr2=0x7f176406c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:41.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.636+0000 7f1768d76700 1 -- 172.21.15.143:0/625225959 shutdown_connections 2023-09-29T23:32:41.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:41.636+0000 7f1768d76700 1 -- 172.21.15.143:0/625225959 wait complete. 2023-09-29T23:32:44.649 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:44.757 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f1c1b700 1 Processor -- start 2023-09-29T23:32:44.758 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f1c1b700 1 -- start start 2023-09-29T23:32:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f1c1b700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f18ec0a9c70 0x7f18ec0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f1c1b700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f18ec1465f0 0x7f18ec1489d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f1c1b700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec0a90a0 con 0x7f18ec0a98a0 2023-09-29T23:32:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f1c1b700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec0b4430 con 0x7f18ec1465f0 2023-09-29T23:32:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f1c1b700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec0aa270 con 0x7f18ec0a9c70 2023-09-29T23:32:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f141a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f18ec1465f0 0x7f18ec1489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f141a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f18ec1465f0 0x7f18ec1489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47364/0 (socket says 172.21.15.143:47364) 2023-09-29T23:32:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f141a700 1 -- 172.21.15.143:0/1439888285 learned_addr learned my addr 172.21.15.143:0/1439888285 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:44.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18ebfff700 1 --2- 172.21.15.143:0/1439888285 >> v2:172.21.15.156:3300/0 conn(0x7f18ec0a9c70 0x7f18ec0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f141a700 1 -- 172.21.15.143:0/1439888285 >> v2:172.21.15.156:3300/0 conn(0x7f18ec0a9c70 msgr2=0x7f18ec0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:44.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f141a700 1 --2- 172.21.15.143:0/1439888285 >> v2:172.21.15.156:3300/0 conn(0x7f18ec0a9c70 0x7f18ec0b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f141a700 1 -- 172.21.15.143:0/1439888285 >> v1:172.21.15.156:6789/0 conn(0x7f18ec0a98a0 legacy=0x7f18ec13d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f141a700 1 -- 172.21.15.143:0/1439888285 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec13e030 con 0x7f18ec1465f0 2023-09-29T23:32:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18ebfff700 1 --2- 172.21.15.143:0/1439888285 >> v2:172.21.15.156:3300/0 conn(0x7f18ec0a9c70 0x7f18ec0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.756+0000 7f18f141a700 1 --2- 172.21.15.143:0/1439888285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18ec1465f0 0x7f18ec1489d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0b3ab0 tx=0x7f18e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e88930436b62dc6 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18eb7fe700 1 -- 172.21.15.143:0/1439888285 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4009e70 con 0x7f18ec1465f0 2023-09-29T23:32:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18eb7fe700 1 -- 172.21.15.143:0/1439888285 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e400b860 con 0x7f18ec1465f0 2023-09-29T23:32:44.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18eb7fe700 1 -- 172.21.15.143:0/1439888285 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e400bc00 con 0x7f18ec1465f0 2023-09-29T23:32:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18f1c1b700 1 -- 172.21.15.143:0/1439888285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18ec1465f0 msgr2=0x7f18ec1489d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18f1c1b700 1 --2- 172.21.15.143:0/1439888285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18ec1465f0 0x7f18ec1489d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f18ec0b3ab0 tx=0x7f18e4009250 comp rx=0 tx=0).stop 2023-09-29T23:32:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18f1c1b700 1 -- 172.21.15.143:0/1439888285 shutdown_connections 2023-09-29T23:32:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18f1c1b700 1 --2- 172.21.15.143:0/1439888285 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18ec1465f0 0x7f18ec1489d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18f1c1b700 1 --2- 172.21.15.143:0/1439888285 >> v2:172.21.15.156:3300/0 conn(0x7f18ec0a9c70 0x7f18ec0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:44.762 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18f1c1b700 1 -- 172.21.15.143:0/1439888285 >> 172.21.15.143:0/1439888285 conn(0x7f18ec0b4730 msgr2=0x7f18ec0b4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:44.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18f1c1b700 1 -- 172.21.15.143:0/1439888285 shutdown_connections 2023-09-29T23:32:44.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18f1c1b700 1 -- 172.21.15.143:0/1439888285 wait complete. 2023-09-29T23:32:44.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18f1c1b700 1 Processor -- start 2023-09-29T23:32:44.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.760+0000 7f18f1c1b700 1 -- start start 2023-09-29T23:32:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f1c1b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f18ec0a98a0 0x7f18ec140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f1c1b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18ec0a9c70 0x7f18ec0ab380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f1c1b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18ec1465f0 0x7f18ec0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f1c1b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec143f80 con 0x7f18ec1465f0 2023-09-29T23:32:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f1c1b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec144240 con 0x7f18ec0a9c70 2023-09-29T23:32:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f1c1b700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec144ec0 con 0x7f18ec0a98a0 2023-09-29T23:32:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f141a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18ec1465f0 0x7f18ec0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f0c19700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f18ec0a98a0 0x7f18ec140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:44.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f141a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18ec1465f0 0x7f18ec0ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47376/0 (socket says 172.21.15.143:47376) 2023-09-29T23:32:44.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f0c19700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f18ec0a98a0 0x7f18ec140a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43404/0 (socket says 172.21.15.143:43404) 2023-09-29T23:32:44.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f141a700 1 -- 172.21.15.143:0/4152813166 learned_addr learned my addr 172.21.15.143:0/4152813166 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:44.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18ebfff700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18ec0a9c70 0x7f18ec0ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:44.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f141a700 1 -- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f18ec0a98a0 msgr2=0x7f18ec140a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:44.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f141a700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f18ec0a98a0 0x7f18ec140a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:44.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f141a700 1 -- 172.21.15.143:0/4152813166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18ec0a9c70 msgr2=0x7f18ec0ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:44.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f141a700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18ec0a9c70 0x7f18ec0ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:44.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f141a700 1 -- 172.21.15.143:0/4152813166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d4009420 con 0x7f18ec1465f0 2023-09-29T23:32:44.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f0c19700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f18ec0a98a0 0x7f18ec140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:44.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18ebfff700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18ec0a9c70 0x7f18ec0ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:44.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f141a700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18ec1465f0 0x7f18ec0ab8c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18e4009290 tx=0x7f18e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:44.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18e8ff9700 1 -- 172.21.15.143:0/4152813166 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e400b9d0 con 0x7f18ec1465f0 2023-09-29T23:32:44.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18e8ff9700 1 -- 172.21.15.143:0/4152813166 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e400bb30 con 0x7f18ec1465f0 2023-09-29T23:32:44.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f1c1b700 1 -- 172.21.15.143:0/4152813166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e40094f0 con 0x7f18ec1465f0 2023-09-29T23:32:44.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18e8ff9700 1 -- 172.21.15.143:0/4152813166 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e4025670 con 0x7f18ec1465f0 2023-09-29T23:32:44.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f1c1b700 1 -- 172.21.15.143:0/4152813166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec135d70 con 0x7f18ec1465f0 2023-09-29T23:32:44.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.764+0000 7f18f1c1b700 1 -- 172.21.15.143:0/4152813166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec136760 con 0x7f18ec1465f0 2023-09-29T23:32:44.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.768+0000 7f18e8ff9700 1 -- 172.21.15.143:0/4152813166 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f18e4023070 con 0x7f18ec1465f0 2023-09-29T23:32:44.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.768+0000 7f18e8ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.768+0000 7f18e8ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.768+0000 7f18e8ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.768+0000 7f18e8ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.768+0000 7f18e8ff9700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f18cc08e8b0 0x7f18cc090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.768+0000 7f18f0c19700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f18cc08e8b0 0x7f18cc090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:44.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.768+0000 7f18e8ff9700 1 -- 172.21.15.143:0/4152813166 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f18e40b5ec0 con 0x7f18ec1465f0 2023-09-29T23:32:44.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.768+0000 7f18f0c19700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f18cc08e8b0 0x7f18cc090d60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18dc007fd0 tx=0x7f18dc012040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:44.775 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.772+0000 7f18e8ff9700 1 -- 172.21.15.143:0/4152813166 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f18ec136760 con 0x7f18ec1465f0 2023-09-29T23:32:44.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.972+0000 7f18f1c1b700 1 -- 172.21.15.143:0/4152813166 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f18ec136760 con 0x7f18ec1465f0 2023-09-29T23:32:44.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.972+0000 7f18e8ff9700 1 -- 172.21.15.143:0/4152813166 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f18e402a100 con 0x7f18ec1465f0 2023-09-29T23:32:44.975 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:44.975 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:44.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 10 mgrc shutdown 2023-09-29T23:32:44.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 -- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f18cc08e8b0 msgr2=0x7f18cc090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:44.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f18cc08e8b0 0x7f18cc090d60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f18dc007fd0 tx=0x7f18dc012040 comp rx=0 tx=0).stop 2023-09-29T23:32:44.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 -- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18ec1465f0 msgr2=0x7f18ec0ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:44.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18ec1465f0 0x7f18ec0ab8c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f18e4009290 tx=0x7f18e4017000 comp rx=0 tx=0).stop 2023-09-29T23:32:44.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 -- 172.21.15.143:0/4152813166 shutdown_connections 2023-09-29T23:32:44.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f18cc08e8b0 0x7f18cc090d60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:44.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f18ec1465f0 0x7f18ec0ab8c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:44.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18ec0a9c70 0x7f18ec0ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:44.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 --2- 172.21.15.143:0/4152813166 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f18ec0a98a0 0x7f18ec140a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:44.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 -- 172.21.15.143:0/4152813166 >> 172.21.15.143:0/4152813166 conn(0x7f18ec0b4730 msgr2=0x7f18ec06fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:44.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 -- 172.21.15.143:0/4152813166 shutdown_connections 2023-09-29T23:32:44.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:44.976+0000 7f18f1c1b700 1 -- 172.21.15.143:0/4152813166 wait complete. 2023-09-29T23:32:47.990 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:48.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f70723b2700 1 Processor -- start 2023-09-29T23:32:48.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f70723b2700 1 -- start start 2023-09-29T23:32:48.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f70723b2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f706c132640 0x7f706c067520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f70723b2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f706c06be10 0x7f706c146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f70723b2700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706c06c240 con 0x7f706c06ba00 2023-09-29T23:32:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f70723b2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706c0a0460 con 0x7f706c06be10 2023-09-29T23:32:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f70723b2700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706c09fc60 con 0x7f706c132640 2023-09-29T23:32:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f7071bb1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f706c06be10 0x7f706c146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f7071bb1700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f706c06be10 0x7f706c146bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47388/0 (socket says 172.21.15.143:47388) 2023-09-29T23:32:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f7071bb1700 1 -- 172.21.15.143:0/1586068977 learned_addr learned my addr 172.21.15.143:0/1586068977 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f70713b0700 1 --2- 172.21.15.143:0/1586068977 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f706c132640 0x7f706c067520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f7071bb1700 1 -- 172.21.15.143:0/1586068977 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f706c132640 msgr2=0x7f706c067520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:48.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f7071bb1700 1 --2- 172.21.15.143:0/1586068977 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f706c132640 0x7f706c067520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:48.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f7071bb1700 1 -- 172.21.15.143:0/1586068977 >> v1:172.21.15.143:6789/0 conn(0x7f706c06ba00 legacy=0x7f706c09f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:48.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f706bfff700 1 -- 172.21.15.143:0/1586068977 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908827453 0 0) 0x7f706c06c240 con 0x7f706c06ba00 2023-09-29T23:32:48.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.100+0000 7f7071bb1700 1 -- 172.21.15.143:0/1586068977 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c09fee0 con 0x7f706c06be10 2023-09-29T23:32:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f7071bb1700 1 --2- 172.21.15.143:0/1586068977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06be10 0x7f706c146bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f706c0a05a0 tx=0x7f7064009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8d17307da747f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f706bfff700 1 -- 172.21.15.143:0/1586068977 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064009e70 con 0x7f706c06be10 2023-09-29T23:32:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f706bfff700 1 -- 172.21.15.143:0/1586068977 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706400b8f0 con 0x7f706c06be10 2023-09-29T23:32:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f706bfff700 1 -- 172.21.15.143:0/1586068977 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706400bcb0 con 0x7f706c06be10 2023-09-29T23:32:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 -- 172.21.15.143:0/1586068977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06be10 msgr2=0x7f706c146bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 --2- 172.21.15.143:0/1586068977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06be10 0x7f706c146bb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f706c0a05a0 tx=0x7f7064009250 comp rx=0 tx=0).stop 2023-09-29T23:32:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 -- 172.21.15.143:0/1586068977 shutdown_connections 2023-09-29T23:32:48.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 --2- 172.21.15.143:0/1586068977 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c06be10 0x7f706c146bb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:48.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 --2- 172.21.15.143:0/1586068977 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f706c132640 0x7f706c067520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:48.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 -- 172.21.15.143:0/1586068977 >> 172.21.15.143:0/1586068977 conn(0x7f706c06f200 msgr2=0x7f706c069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:48.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 -- 172.21.15.143:0/1586068977 shutdown_connections 2023-09-29T23:32:48.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 -- 172.21.15.143:0/1586068977 wait complete. 2023-09-29T23:32:48.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 Processor -- start 2023-09-29T23:32:48.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 -- start start 2023-09-29T23:32:48.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f706c06ba00 0x7f706c145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:48.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f706c06be10 0x7f706c12f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c132640 0x7f706c153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c145cd0 con 0x7f706c132640 2023-09-29T23:32:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c1462e0 con 0x7f706c06ba00 2023-09-29T23:32:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70723b2700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706c153020 con 0x7f706c06be10 2023-09-29T23:32:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f7071bb1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c132640 0x7f706c153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f7070baf700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f706c06be10 0x7f706c12f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f7071bb1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c132640 0x7f706c153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47392/0 (socket says 172.21.15.143:47392) 2023-09-29T23:32:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f7071bb1700 1 -- 172.21.15.143:0/3985729248 learned_addr learned my addr 172.21.15.143:0/3985729248 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f70713b0700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f706c06ba00 0x7f706c145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f7071bb1700 1 -- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f706c06be10 msgr2=0x7f706c12f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:48.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.104+0000 7f7071bb1700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f706c06be10 0x7f706c12f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f7071bb1700 1 -- 172.21.15.143:0/3985729248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f706c06ba00 msgr2=0x7f706c145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f7071bb1700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f706c06ba00 0x7f706c145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f7071bb1700 1 -- 172.21.15.143:0/3985729248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70640094f0 con 0x7f706c132640 2023-09-29T23:32:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f7070baf700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f706c06be10 0x7f706c12f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f70713b0700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f706c06ba00 0x7f706c145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:32:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f7071bb1700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c132640 0x7f706c153500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7064009290 tx=0x7f7064016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f70697fa700 1 -- 172.21.15.143:0/3985729248 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064011ea0 con 0x7f706c132640 2023-09-29T23:32:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f70697fa700 1 -- 172.21.15.143:0/3985729248 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7064020070 con 0x7f706c132640 2023-09-29T23:32:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f70697fa700 1 -- 172.21.15.143:0/3985729248 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706401b6a0 con 0x7f706c132640 2023-09-29T23:32:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f70723b2700 1 -- 172.21.15.143:0/3985729248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706c146580 con 0x7f706c132640 2023-09-29T23:32:48.111 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f70723b2700 1 -- 172.21.15.143:0/3985729248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c141e50 con 0x7f706c132640 2023-09-29T23:32:48.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f70723b2700 1 -- 172.21.15.143:0/3985729248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706c05ecb0 con 0x7f706c132640 2023-09-29T23:32:48.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f70697fa700 1 -- 172.21.15.143:0/3985729248 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f7064022030 con 0x7f706c132640 2023-09-29T23:32:48.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.108+0000 7f70697fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:48.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.112+0000 7f70697fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:48.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.112+0000 7f70697fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:48.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.112+0000 7f70697fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:48.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.112+0000 7f70697fa700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f705808e8e0 0x7f7058090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:48.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.112+0000 7f70697fa700 1 -- 172.21.15.143:0/3985729248 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f70640b62d0 con 0x7f706c132640 2023-09-29T23:32:48.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.112+0000 7f70713b0700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f705808e8e0 0x7f7058090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:48.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.112+0000 7f70713b0700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f705808e8e0 0x7f7058090d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7060009d80 tx=0x7f7060008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:48.119 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.116+0000 7f70697fa700 1 -- 172.21.15.143:0/3985729248 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7064086d20 con 0x7f706c132640 2023-09-29T23:32:48.316 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 -- 172.21.15.143:0/3985729248 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f706c0a01a0 con 0x7f706c132640 2023-09-29T23:32:48.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70697fa700 1 -- 172.21.15.143:0/3985729248 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f706402b100 con 0x7f706c132640 2023-09-29T23:32:48.317 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:48.317 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:48.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 10 mgrc shutdown 2023-09-29T23:32:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 -- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f705808e8e0 msgr2=0x7f7058090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f705808e8e0 0x7f7058090d90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7060009d80 tx=0x7f7060008040 comp rx=0 tx=0).stop 2023-09-29T23:32:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 -- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c132640 msgr2=0x7f706c153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c132640 0x7f706c153500 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7064009290 tx=0x7f7064016000 comp rx=0 tx=0).stop 2023-09-29T23:32:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 -- 172.21.15.143:0/3985729248 shutdown_connections 2023-09-29T23:32:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f705808e8e0 0x7f7058090d90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f706c132640 0x7f706c153500 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f706c06be10 0x7f706c12f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 --2- 172.21.15.143:0/3985729248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f706c06ba00 0x7f706c145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:48.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 -- 172.21.15.143:0/3985729248 >> 172.21.15.143:0/3985729248 conn(0x7f706c06f200 msgr2=0x7f706c0a4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 -- 172.21.15.143:0/3985729248 shutdown_connections 2023-09-29T23:32:48.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:48.316+0000 7f70723b2700 1 -- 172.21.15.143:0/3985729248 wait complete. 2023-09-29T23:32:51.331 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:51.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50ffff700 1 Processor -- start 2023-09-29T23:32:51.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50ffff700 1 -- start start 2023-09-29T23:32:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50ffff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb5100a6350 0x7fb51006b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50ffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 0x7fb5101363f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:51.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50ffff700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb510136930 0x7fb5100a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50ffff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5100a5ed0 con 0x7fb5100a6350 2023-09-29T23:32:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50ffff700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5100a6720 con 0x7fb510136930 2023-09-29T23:32:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50ffff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100a5c10 con 0x7fb51006b8a0 2023-09-29T23:32:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb507fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 0x7fb5101363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50effd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb5100a6350 0x7fb51006b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb507fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 0x7fb5101363f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43434/0 (socket says 172.21.15.143:43434) 2023-09-29T23:32:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb507fff700 1 -- 172.21.15.143:0/1785203756 learned_addr learned my addr 172.21.15.143:0/1785203756 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:51.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50f7fe700 1 --2- 172.21.15.143:0/1785203756 >> v2:172.21.15.156:3300/0 conn(0x7fb510136930 0x7fb5100a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb507fff700 1 -- 172.21.15.143:0/1785203756 >> v2:172.21.15.156:3300/0 conn(0x7fb510136930 msgr2=0x7fb5100a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb507fff700 1 --2- 172.21.15.143:0/1785203756 >> v2:172.21.15.156:3300/0 conn(0x7fb510136930 0x7fb5100a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb507fff700 1 -- 172.21.15.143:0/1785203756 >> v2:172.21.15.143:3300/0 conn(0x7fb5100a6350 msgr2=0x7fb51006b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb507fff700 1 --2- 172.21.15.143:0/1785203756 >> v2:172.21.15.143:3300/0 conn(0x7fb5100a6350 0x7fb51006b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb507fff700 1 -- 172.21.15.143:0/1785203756 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100a94e0 con 0x7fb51006b8a0 2023-09-29T23:32:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50f7fe700 1 --2- 172.21.15.143:0/1785203756 >> v2:172.21.15.156:3300/0 conn(0x7fb510136930 0x7fb5100a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.444+0000 7fb50effd700 1 --2- 172.21.15.143:0/1785203756 >> v2:172.21.15.143:3300/0 conn(0x7fb5100a6350 0x7fb51006b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb507fff700 1 --2- 172.21.15.143:0/1785203756 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 0x7fb5101363f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb5100a9dd0 tx=0x7fb500009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16af48dfc0f609ac server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50e7fc700 1 -- 172.21.15.143:0/1785203756 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50000b640 con 0x7fb51006b8a0 2023-09-29T23:32:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50e7fc700 1 -- 172.21.15.143:0/1785203756 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50000b7a0 con 0x7fb51006b8a0 2023-09-29T23:32:51.450 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50e7fc700 1 -- 172.21.15.143:0/1785203756 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50000bb60 con 0x7fb51006b8a0 2023-09-29T23:32:51.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 -- 172.21.15.143:0/1785203756 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 msgr2=0x7fb5101363f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:51.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 --2- 172.21.15.143:0/1785203756 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 0x7fb5101363f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb5100a9dd0 tx=0x7fb500009250 comp rx=0 tx=0).stop 2023-09-29T23:32:51.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 -- 172.21.15.143:0/1785203756 shutdown_connections 2023-09-29T23:32:51.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 --2- 172.21.15.143:0/1785203756 >> v2:172.21.15.156:3300/0 conn(0x7fb510136930 0x7fb5100a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 --2- 172.21.15.143:0/1785203756 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 0x7fb5101363f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 --2- 172.21.15.143:0/1785203756 >> v2:172.21.15.143:3300/0 conn(0x7fb5100a6350 0x7fb51006b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 -- 172.21.15.143:0/1785203756 >> 172.21.15.143:0/1785203756 conn(0x7fb5100b6930 msgr2=0x7fb5100b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:51.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 -- 172.21.15.143:0/1785203756 shutdown_connections 2023-09-29T23:32:51.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 -- 172.21.15.143:0/1785203756 wait complete. 2023-09-29T23:32:51.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 Processor -- start 2023-09-29T23:32:51.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 -- start start 2023-09-29T23:32:51.453 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 0x7fb5100b4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5100a6350 0x7fb5100b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb510136930 0x7fb51012fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100b4420 con 0x7fb510136930 2023-09-29T23:32:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb510142220 con 0x7fb5100a6350 2023-09-29T23:32:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50ffff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5101424e0 con 0x7fb51006b8a0 2023-09-29T23:32:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50effd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 0x7fb5100b4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb510136930 0x7fb51012fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb510136930 0x7fb51012fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:47408/0 (socket says 172.21.15.143:47408) 2023-09-29T23:32:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50f7fe700 1 -- 172.21.15.143:0/3704308020 learned_addr learned my addr 172.21.15.143:0/3704308020 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50effd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 0x7fb5100b4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43438/0 (socket says 172.21.15.143:43438) 2023-09-29T23:32:51.454 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb507fff700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5100a6350 0x7fb5100b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.448+0000 7fb50f7fe700 1 -- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 msgr2=0x7fb5100b4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.452+0000 7fb50f7fe700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 0x7fb5100b4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.452+0000 7fb50f7fe700 1 -- 172.21.15.143:0/3704308020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5100a6350 msgr2=0x7fb5100b4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.452+0000 7fb50f7fe700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5100a6350 0x7fb5100b4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.452+0000 7fb50f7fe700 1 -- 172.21.15.143:0/3704308020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f8009420 con 0x7fb510136930 2023-09-29T23:32:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.452+0000 7fb50f7fe700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb510136930 0x7fb51012fe70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb508009f50 tx=0x7fb50800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.452+0000 7fb506ffd700 1 -- 172.21.15.143:0/3704308020 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800caf0 con 0x7fb510136930 2023-09-29T23:32:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.452+0000 7fb506ffd700 1 -- 172.21.15.143:0/3704308020 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb50800f040 con 0x7fb510136930 2023-09-29T23:32:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.452+0000 7fb50ffff700 1 -- 172.21.15.143:0/3704308020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb508009a20 con 0x7fb510136930 2023-09-29T23:32:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.452+0000 7fb506ffd700 1 -- 172.21.15.143:0/3704308020 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50800ce80 con 0x7fb510136930 2023-09-29T23:32:51.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.452+0000 7fb50ffff700 1 -- 172.21.15.143:0/3704308020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5000094f0 con 0x7fb510136930 2023-09-29T23:32:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.452+0000 7fb50ffff700 1 -- 172.21.15.143:0/3704308020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5100b4560 con 0x7fb510136930 2023-09-29T23:32:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.456+0000 7fb506ffd700 1 -- 172.21.15.143:0/3704308020 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fb508014690 con 0x7fb510136930 2023-09-29T23:32:51.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.456+0000 7fb506ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:51.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.456+0000 7fb506ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:51.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.456+0000 7fb506ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:51.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.456+0000 7fb506ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:51.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.456+0000 7fb506ffd700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4f008e8b0 0x7fb4f0090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:51.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.456+0000 7fb50effd700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4f008e8b0 0x7fb4f0090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:51.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.456+0000 7fb506ffd700 1 -- 172.21.15.143:0/3704308020 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fb5080b54b0 con 0x7fb510136930 2023-09-29T23:32:51.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.456+0000 7fb50effd700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4f008e8b0 0x7fb4f0090d60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009f40 tx=0x7fb4f80091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:51.465 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.464+0000 7fb506ffd700 1 -- 172.21.15.143:0/3704308020 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb5100b4560 con 0x7fb510136930 2023-09-29T23:32:51.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 -- 172.21.15.143:0/3704308020 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb5100b4560 con 0x7fb510136930 2023-09-29T23:32:51.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb506ffd700 1 -- 172.21.15.143:0/3704308020 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fb5100b4560 con 0x7fb510136930 2023-09-29T23:32:51.665 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:51.665 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:51.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 10 mgrc shutdown 2023-09-29T23:32:51.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 -- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4f008e8b0 msgr2=0x7fb4f0090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:51.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4f008e8b0 0x7fb4f0090d60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8009f40 tx=0x7fb4f80091a0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 -- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb510136930 msgr2=0x7fb51012fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:51.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb510136930 0x7fb51012fe70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb508009f50 tx=0x7fb50800b040 comp rx=0 tx=0).stop 2023-09-29T23:32:51.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 -- 172.21.15.143:0/3704308020 shutdown_connections 2023-09-29T23:32:51.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4f008e8b0 0x7fb4f0090d60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb510136930 0x7fb51012fe70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5100a6350 0x7fb5100b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 --2- 172.21.15.143:0/3704308020 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb51006b8a0 0x7fb5100b4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:51.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 -- 172.21.15.143:0/3704308020 >> 172.21.15.143:0/3704308020 conn(0x7fb5100b6930 msgr2=0x7fb5100b5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:51.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 -- 172.21.15.143:0/3704308020 shutdown_connections 2023-09-29T23:32:51.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:51.664+0000 7fb50ffff700 1 -- 172.21.15.143:0/3704308020 wait complete. 2023-09-29T23:32:54.685 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:54.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.796+0000 7fb264ecd700 1 Processor -- start 2023-09-29T23:32:54.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.796+0000 7fb264ecd700 1 -- start start 2023-09-29T23:32:54.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.796+0000 7fb264ecd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2600a6350 0x7fb26006b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:54.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.796+0000 7fb264ecd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb26006b8a0 0x7fb2601363f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:54.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.796+0000 7fb264ecd700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2600a5ed0 con 0x7fb260136930 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.796+0000 7fb264ecd700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2600a6720 con 0x7fb26006b8a0 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.796+0000 7fb264ecd700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2600a5c10 con 0x7fb2600a6350 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.796+0000 7fb25f7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2600a6350 0x7fb26006b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.796+0000 7fb25effd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb26006b8a0 0x7fb2601363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.796+0000 7fb25effd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb26006b8a0 0x7fb2601363f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58614/0 (socket says 172.21.15.143:58614) 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.796+0000 7fb25effd700 1 -- 172.21.15.143:0/203645472 learned_addr learned my addr 172.21.15.143:0/203645472 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb25f7fe700 1 -- 172.21.15.143:0/203645472 >> v2:172.21.15.143:3300/0 conn(0x7fb26006b8a0 msgr2=0x7fb2601363f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb25e7fc700 1 -- 172.21.15.143:0/203645472 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1685787209 0 0) 0x7fb2600a5ed0 con 0x7fb260136930 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb25f7fe700 1 --2- 172.21.15.143:0/203645472 >> v2:172.21.15.143:3300/0 conn(0x7fb26006b8a0 0x7fb2601363f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb25f7fe700 1 -- 172.21.15.143:0/203645472 >> v1:172.21.15.143:6789/0 conn(0x7fb260136930 legacy=0x7fb2600a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb25f7fe700 1 -- 172.21.15.143:0/203645472 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2600a94e0 con 0x7fb2600a6350 2023-09-29T23:32:54.801 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb25effd700 1 --2- 172.21.15.143:0/203645472 >> v2:172.21.15.143:3300/0 conn(0x7fb26006b8a0 0x7fb2601363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:32:54.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb25f7fe700 1 --2- 172.21.15.143:0/203645472 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2600a6350 0x7fb26006b360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2540098e0 tx=0x7fb2540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae419778ef7b73e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:54.802 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb25e7fc700 1 -- 172.21.15.143:0/203645472 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400b640 con 0x7fb2600a6350 2023-09-29T23:32:54.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb25e7fc700 1 -- 172.21.15.143:0/203645472 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25400b7a0 con 0x7fb2600a6350 2023-09-29T23:32:54.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb25e7fc700 1 -- 172.21.15.143:0/203645472 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25400bb40 con 0x7fb2600a6350 2023-09-29T23:32:54.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb264ecd700 1 -- 172.21.15.143:0/203645472 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2600a6350 msgr2=0x7fb26006b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:54.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb264ecd700 1 --2- 172.21.15.143:0/203645472 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2600a6350 0x7fb26006b360 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2540098e0 tx=0x7fb2540091a0 comp rx=0 tx=0).stop 2023-09-29T23:32:54.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb264ecd700 1 -- 172.21.15.143:0/203645472 shutdown_connections 2023-09-29T23:32:54.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb264ecd700 1 --2- 172.21.15.143:0/203645472 >> v2:172.21.15.143:3300/0 conn(0x7fb26006b8a0 0x7fb2601363f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:54.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb264ecd700 1 --2- 172.21.15.143:0/203645472 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb2600a6350 0x7fb26006b360 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:54.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb264ecd700 1 -- 172.21.15.143:0/203645472 >> 172.21.15.143:0/203645472 conn(0x7fb2600b6930 msgr2=0x7fb2600b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:54.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb264ecd700 1 -- 172.21.15.143:0/203645472 shutdown_connections 2023-09-29T23:32:54.803 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb264ecd700 1 -- 172.21.15.143:0/203645472 wait complete. 2023-09-29T23:32:54.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb264ecd700 1 Processor -- start 2023-09-29T23:32:54.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.800+0000 7fb264ecd700 1 -- start start 2023-09-29T23:32:54.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb264ecd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb26006b8a0 0x7fb260134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:54.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb264ecd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb260136930 0x7fb260134770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:54.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb264ecd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb26012fa10 0x7fb26012fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:54.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb264ecd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb260133ac0 con 0x7fb26012fa10 2023-09-29T23:32:54.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb264ecd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb260142eb0 con 0x7fb26006b8a0 2023-09-29T23:32:54.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb264ecd700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb260142bd0 con 0x7fb260136930 2023-09-29T23:32:54.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25effd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb260136930 0x7fb260134770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:54.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25effd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb260136930 0x7fb260134770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:55234/0 (socket says 172.21.15.143:55234) 2023-09-29T23:32:54.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25effd700 1 -- 172.21.15.143:0/432213060 learned_addr learned my addr 172.21.15.143:0/432213060 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:54.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25ffff700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb26012fa10 0x7fb26012fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:54.806 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25f7fe700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb26006b8a0 0x7fb260134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:54.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25effd700 1 -- 172.21.15.143:0/432213060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb26006b8a0 msgr2=0x7fb260134230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:54.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25effd700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb26006b8a0 0x7fb260134230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:54.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25effd700 1 -- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb26012fa10 msgr2=0x7fb26012fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:54.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25effd700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb26012fa10 0x7fb26012fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:54.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25effd700 1 -- 172.21.15.143:0/432213060 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb250009ae0 con 0x7fb260136930 2023-09-29T23:32:54.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25f7fe700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb26006b8a0 0x7fb260134230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:32:54.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb25effd700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb260136930 0x7fb260134770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb25000c170 tx=0x7fb25000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:54.807 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb2477fe700 1 -- 172.21.15.143:0/432213060 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25000cb50 con 0x7fb260136930 2023-09-29T23:32:54.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb2477fe700 1 -- 172.21.15.143:0/432213060 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25000f040 con 0x7fb260136930 2023-09-29T23:32:54.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb264ecd700 1 -- 172.21.15.143:0/432213060 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb254009420 con 0x7fb260136930 2023-09-29T23:32:54.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb2477fe700 1 -- 172.21.15.143:0/432213060 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb250014480 con 0x7fb260136930 2023-09-29T23:32:54.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.804+0000 7fb264ecd700 1 -- 172.21.15.143:0/432213060 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26012f470 con 0x7fb260136930 2023-09-29T23:32:54.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.808+0000 7fb264ecd700 1 -- 172.21.15.143:0/432213060 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26005ecb0 con 0x7fb260136930 2023-09-29T23:32:54.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.808+0000 7fb2477fe700 1 -- 172.21.15.143:0/432213060 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fb2500147f0 con 0x7fb260136930 2023-09-29T23:32:54.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.808+0000 7fb2477fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:54.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.808+0000 7fb2477fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:54.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.808+0000 7fb2477fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:54.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.808+0000 7fb2477fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:54.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.808+0000 7fb2477fe700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb24c08e8b0 0x7fb24c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:54.810 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.808+0000 7fb2477fe700 1 -- 172.21.15.143:0/432213060 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fb2500b4f50 con 0x7fb260136930 2023-09-29T23:32:54.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.808+0000 7fb25f7fe700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb24c08e8b0 0x7fb24c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:54.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.808+0000 7fb25f7fe700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb24c08e8b0 0x7fb24c090d60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb2600b7a00 tx=0x7fb254017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:54.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:54.816+0000 7fb2477fe700 1 -- 172.21.15.143:0/432213060 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb2500859a0 con 0x7fb260136930 2023-09-29T23:32:55.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.004+0000 7fb264ecd700 1 -- 172.21.15.143:0/432213060 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb26004b5c0 con 0x7fb260136930 2023-09-29T23:32:55.008 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.008+0000 7fb2477fe700 1 -- 172.21.15.143:0/432213060 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fb250088fc0 con 0x7fb260136930 2023-09-29T23:32:55.009 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:55.009 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.008+0000 7fb264ecd700 10 mgrc shutdown 2023-09-29T23:32:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.008+0000 7fb264ecd700 1 -- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb24c08e8b0 msgr2=0x7fb24c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:55.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.008+0000 7fb264ecd700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb24c08e8b0 0x7fb24c090d60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb2600b7a00 tx=0x7fb254017000 comp rx=0 tx=0).stop 2023-09-29T23:32:55.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.008+0000 7fb264ecd700 1 -- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb260136930 msgr2=0x7fb260134770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:55.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.008+0000 7fb264ecd700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb260136930 0x7fb260134770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb25000c170 tx=0x7fb25000b040 comp rx=0 tx=0).stop 2023-09-29T23:32:55.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.008+0000 7fb264ecd700 1 -- 172.21.15.143:0/432213060 shutdown_connections 2023-09-29T23:32:55.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.012+0000 7fb264ecd700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb24c08e8b0 0x7fb24c090d60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:55.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.012+0000 7fb264ecd700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb26012fa10 0x7fb26012fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:55.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.012+0000 7fb264ecd700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb260136930 0x7fb260134770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:55.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.012+0000 7fb264ecd700 1 --2- 172.21.15.143:0/432213060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb26006b8a0 0x7fb260134230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:55.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.012+0000 7fb264ecd700 1 -- 172.21.15.143:0/432213060 >> 172.21.15.143:0/432213060 conn(0x7fb2600b6930 msgr2=0x7fb26006caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:55.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.012+0000 7fb264ecd700 1 -- 172.21.15.143:0/432213060 shutdown_connections 2023-09-29T23:32:55.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:55.012+0000 7fb264ecd700 1 -- 172.21.15.143:0/432213060 wait complete. 2023-09-29T23:32:56.206 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:32:56.208 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:32:58.029 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:32:58.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.140+0000 7f2f559a0700 1 Processor -- start 2023-09-29T23:32:58.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 -- start start 2023-09-29T23:32:58.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f500b4210 0x7f2f500a9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:58.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2f5013cb50 0x7f2f5013cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:58.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f500b45e0 con 0x7f2f50066c90 2023-09-29T23:32:58.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5013dc00 con 0x7f2f500b4210 2023-09-29T23:32:58.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f500673a0 con 0x7f2f5013cb50 2023-09-29T23:32:58.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f5499e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2f500b4210 0x7f2f500a9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:58.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f4ffff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2f50066c90 0x7f2f500a86f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37028/0 (socket says 172.21.15.143:37028) 2023-09-29T23:32:58.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f4ffff700 1 -- 172.21.15.143:0/2163095979 learned_addr learned my addr 172.21.15.143:0/2163095979 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:32:58.145 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f5519f700 1 --2- 172.21.15.143:0/2163095979 >> v2:172.21.15.156:3300/0 conn(0x7f2f5013cb50 0x7f2f5013cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:58.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f5499e700 1 -- 172.21.15.143:0/2163095979 >> v2:172.21.15.156:3300/0 conn(0x7f2f5013cb50 msgr2=0x7f2f5013cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:32:58.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f5499e700 1 --2- 172.21.15.143:0/2163095979 >> v2:172.21.15.156:3300/0 conn(0x7f2f5013cb50 0x7f2f5013cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:58.146 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f5499e700 1 -- 172.21.15.143:0/2163095979 >> v1:172.21.15.143:6789/0 conn(0x7f2f50066c90 legacy=0x7f2f500a86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:58.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f5499e700 1 -- 172.21.15.143:0/2163095979 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f50067660 con 0x7f2f500b4210 2023-09-29T23:32:58.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f5499e700 1 --2- 172.21.15.143:0/2163095979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f500b4210 0x7f2f500a9fa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f500a78f0 tx=0x7f2f440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391ecaaf449f231f server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:58.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f4f7fe700 1 -- 172.21.15.143:0/2163095979 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44011630 con 0x7f2f500b4210 2023-09-29T23:32:58.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f4f7fe700 1 -- 172.21.15.143:0/2163095979 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f44011790 con 0x7f2f500b4210 2023-09-29T23:32:58.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f4f7fe700 1 -- 172.21.15.143:0/2163095979 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44011b30 con 0x7f2f500b4210 2023-09-29T23:32:58.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 -- 172.21.15.143:0/2163095979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f500b4210 msgr2=0x7f2f500a9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:58.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 --2- 172.21.15.143:0/2163095979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f500b4210 0x7f2f500a9fa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f500a78f0 tx=0x7f2f440091a0 comp rx=0 tx=0).stop 2023-09-29T23:32:58.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 -- 172.21.15.143:0/2163095979 shutdown_connections 2023-09-29T23:32:58.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 --2- 172.21.15.143:0/2163095979 >> v2:172.21.15.156:3300/0 conn(0x7f2f5013cb50 0x7f2f5013cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:58.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 --2- 172.21.15.143:0/2163095979 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f500b4210 0x7f2f500a9fa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:58.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 -- 172.21.15.143:0/2163095979 >> 172.21.15.143:0/2163095979 conn(0x7f2f500b4820 msgr2=0x7f2f500b4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:58.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 -- 172.21.15.143:0/2163095979 shutdown_connections 2023-09-29T23:32:58.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.144+0000 7f2f559a0700 1 -- 172.21.15.143:0/2163095979 wait complete. 2023-09-29T23:32:58.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f559a0700 1 Processor -- start 2023-09-29T23:32:58.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f559a0700 1 -- start start 2023-09-29T23:32:58.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f559a0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f50066c90 0x7f2f5006b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:58.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f559a0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f500b4210 0x7f2f500b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:58.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f559a0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2f5013cb50 0x7f2f50141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:58.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f559a0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f50137760 con 0x7f2f50066c90 2023-09-29T23:32:58.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f559a0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f50150f80 con 0x7f2f500b4210 2023-09-29T23:32:58.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f559a0700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f50150410 con 0x7f2f5013cb50 2023-09-29T23:32:58.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f5499e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f50066c90 0x7f2f5006b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:58.150 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f5499e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f50066c90 0x7f2f5006b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58644/0 (socket says 172.21.15.143:58644) 2023-09-29T23:32:58.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f5499e700 1 -- 172.21.15.143:0/3377135623 learned_addr learned my addr 172.21.15.143:0/3377135623 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:32:58.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f5519f700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2f5013cb50 0x7f2f50141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:58.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f4ffff700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f500b4210 0x7f2f500b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:58.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f5499e700 1 -- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2f5013cb50 msgr2=0x7f2f50141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:58.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f5499e700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2f5013cb50 0x7f2f50141990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:58.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f5499e700 1 -- 172.21.15.143:0/3377135623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f500b4210 msgr2=0x7f2f500b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:58.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f5499e700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f500b4210 0x7f2f500b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:58.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f5499e700 1 -- 172.21.15.143:0/3377135623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f44009420 con 0x7f2f50066c90 2023-09-29T23:32:58.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f5499e700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f50066c90 0x7f2f5006b0d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2f50070660 tx=0x7f2f44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:58.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f4cff9700 1 -- 172.21.15.143:0/3377135623 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44000d10 con 0x7f2f50066c90 2023-09-29T23:32:58.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f4cff9700 1 -- 172.21.15.143:0/3377135623 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f44011b10 con 0x7f2f50066c90 2023-09-29T23:32:58.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f559a0700 1 -- 172.21.15.143:0/3377135623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f501506d0 con 0x7f2f50066c90 2023-09-29T23:32:58.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f4cff9700 1 -- 172.21.15.143:0/3377135623 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44025640 con 0x7f2f50066c90 2023-09-29T23:32:58.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.148+0000 7f2f559a0700 1 -- 172.21.15.143:0/3377135623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f5006b610 con 0x7f2f50066c90 2023-09-29T23:32:58.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.152+0000 7f2f559a0700 1 -- 172.21.15.143:0/3377135623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f5005ecb0 con 0x7f2f50066c90 2023-09-29T23:32:58.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.152+0000 7f2f4cff9700 1 -- 172.21.15.143:0/3377135623 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f2f44025a30 con 0x7f2f50066c90 2023-09-29T23:32:58.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.152+0000 7f2f4cff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:32:58.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.152+0000 7f2f4cff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:32:58.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.152+0000 7f2f4cff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:58.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.152+0000 7f2f4cff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:32:58.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.152+0000 7f2f4cff9700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f3c08e8b0 0x7f2f3c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:32:58.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.152+0000 7f2f4ffff700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f3c08e8b0 0x7f2f3c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:32:58.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.152+0000 7f2f4cff9700 1 -- 172.21.15.143:0/3377135623 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f2f440b6930 con 0x7f2f50066c90 2023-09-29T23:32:58.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.156+0000 7f2f4ffff700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f3c08e8b0 0x7f2f3c090d60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2f5013b8e0 tx=0x7f2f40007eb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:32:58.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.160+0000 7f2f4cff9700 1 -- 172.21.15.143:0/3377135623 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2f440872e0 con 0x7f2f50066c90 2023-09-29T23:32:58.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.356+0000 7f2f559a0700 1 -- 172.21.15.143:0/3377135623 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2f500a4a10 con 0x7f2f50066c90 2023-09-29T23:32:58.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.356+0000 7f2f4cff9700 1 -- 172.21.15.143:0/3377135623 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f2f4408a900 con 0x7f2f50066c90 2023-09-29T23:32:58.358 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:32:58.358 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:32:58.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 10 mgrc shutdown 2023-09-29T23:32:58.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 -- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f3c08e8b0 msgr2=0x7f2f3c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:58.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f3c08e8b0 0x7f2f3c090d60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2f5013b8e0 tx=0x7f2f40007eb0 comp rx=0 tx=0).stop 2023-09-29T23:32:58.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 -- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f50066c90 msgr2=0x7f2f5006b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:32:58.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f50066c90 0x7f2f5006b0d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2f50070660 tx=0x7f2f44017000 comp rx=0 tx=0).stop 2023-09-29T23:32:58.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 -- 172.21.15.143:0/3377135623 shutdown_connections 2023-09-29T23:32:58.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f3c08e8b0 0x7f2f3c090d60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:58.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2f5013cb50 0x7f2f50141990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:58.362 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f500b4210 0x7f2f500b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:58.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 --2- 172.21.15.143:0/3377135623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2f50066c90 0x7f2f5006b0d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:32:58.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 -- 172.21.15.143:0/3377135623 >> 172.21.15.143:0/3377135623 conn(0x7f2f500b4820 msgr2=0x7f2f5006fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:32:58.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 -- 172.21.15.143:0/3377135623 shutdown_connections 2023-09-29T23:32:58.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:32:58.360+0000 7f2f559a0700 1 -- 172.21.15.143:0/3377135623 wait complete. 2023-09-29T23:33:01.378 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:01.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.487+0000 7ff5e48dc700 1 Processor -- start 2023-09-29T23:33:01.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.487+0000 7ff5e48dc700 1 -- start start 2023-09-29T23:33:01.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.487+0000 7ff5e48dc700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff5e00a70a0 0x7ff5e00a7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:01.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.487+0000 7ff5e48dc700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e00a4d30 con 0x7ff5e00a33f0 2023-09-29T23:33:01.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.487+0000 7ff5e48dc700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e013eca0 con 0x7ff5e00a2fd0 2023-09-29T23:33:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.487+0000 7ff5e48dc700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e00a2310 con 0x7ff5e00a70a0 2023-09-29T23:33:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.487+0000 7ff5df59e700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7ff5e00a33f0 0x7ff5e00a1680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:37044/0 (socket says 172.21.15.143:37044) 2023-09-29T23:33:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.487+0000 7ff5df59e700 1 -- 172.21.15.143:0/512403789 learned_addr learned my addr 172.21.15.143:0/512403789 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:33:01.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.487+0000 7ff5ded9d700 1 --2- 172.21.15.143:0/512403789 >> v2:172.21.15.156:3300/0 conn(0x7ff5e00a70a0 0x7ff5e00a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:01.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.487+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672079656 0 0) 0x7ff5e00a4d30 con 0x7ff5e00a33f0 2023-09-29T23:33:01.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5cc003570 con 0x7ff5e00a33f0 2023-09-29T23:33:01.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527636931 0 0) 0x7ff5e013eca0 con 0x7ff5e00a2fd0 2023-09-29T23:33:01.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5e00a4d30 con 0x7ff5e00a2fd0 2023-09-29T23:33:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2873616917 0 0) 0x7ff5cc003570 con 0x7ff5e00a33f0 2023-09-29T23:33:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 >> v2:172.21.15.156:3300/0 conn(0x7ff5e00a70a0 msgr2=0x7ff5e00a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 --2- 172.21.15.143:0/512403789 >> v2:172.21.15.156:3300/0 conn(0x7ff5e00a70a0 0x7ff5e00a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 >> v1:172.21.15.156:6789/0 conn(0x7ff5e00a2fd0 legacy=0x7ff5e013e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e00a1d90 con 0x7ff5e00a33f0 2023-09-29T23:33:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251153714 0 0) 0x7ff5d80017f0 con 0x7ff5e00a33f0 2023-09-29T23:33:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 >> v1:172.21.15.143:6789/0 conn(0x7ff5e00a33f0 legacy=0x7ff5e00a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 --2- 172.21.15.143:0/512403789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5cc003bf0 0x7ff5cc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 --2- 172.21.15.143:0/512403789 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5cc004810 0x7ff5cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 --2- 172.21.15.143:0/512403789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5cc0051c0 0x7ff5cc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:01.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e00a2310 con 0x7ff5cc003bf0 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc00c0b0 con 0x7ff5cc0051c0 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc00c270 con 0x7ff5cc004810 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5d7fff700 1 --2- 172.21.15.143:0/512403789 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5cc004810 0x7ff5cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5df59e700 1 --2- 172.21.15.143:0/512403789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5cc0051c0 0x7ff5cc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5ded9d700 1 --2- 172.21.15.143:0/512403789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5cc003bf0 0x7ff5cc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5d7fff700 1 -- 172.21.15.143:0/512403789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5cc0051c0 msgr2=0x7ff5cc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5d7fff700 1 --2- 172.21.15.143:0/512403789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5cc0051c0 0x7ff5cc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5d7fff700 1 -- 172.21.15.143:0/512403789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5cc003bf0 msgr2=0x7ff5cc003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5d7fff700 1 --2- 172.21.15.143:0/512403789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5cc003bf0 0x7ff5cc003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5df59e700 1 -- 172.21.15.143:0/512403789 reap_dead start 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5d7fff700 1 -- 172.21.15.143:0/512403789 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc003570 con 0x7ff5cc004810 2023-09-29T23:33:01.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5d7fff700 1 --2- 172.21.15.143:0/512403789 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5cc004810 0x7ff5cc004c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8001a10 tx=0x7ff5c8007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce25876b4b288b85 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:01.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800c590 con 0x7ff5cc004810 2023-09-29T23:33:01.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c800c6f0 con 0x7ff5cc004810 2023-09-29T23:33:01.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5de59c700 1 -- 172.21.15.143:0/512403789 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800ca90 con 0x7ff5cc004810 2023-09-29T23:33:01.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5e48dc700 1 -- 172.21.15.143:0/512403789 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5cc004810 msgr2=0x7ff5cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:01.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5e48dc700 1 --2- 172.21.15.143:0/512403789 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5cc004810 0x7ff5cc004c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff5c8001a10 tx=0x7ff5c8007d30 comp rx=0 tx=0).stop 2023-09-29T23:33:01.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5e48dc700 1 -- 172.21.15.143:0/512403789 shutdown_connections 2023-09-29T23:33:01.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5e48dc700 1 --2- 172.21.15.143:0/512403789 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5cc004810 0x7ff5cc004c80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:01.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5e48dc700 1 -- 172.21.15.143:0/512403789 >> 172.21.15.143:0/512403789 conn(0x7ff5e00b46e0 msgr2=0x7ff5e00b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:01.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5e48dc700 1 -- 172.21.15.143:0/512403789 shutdown_connections 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.491+0000 7ff5e48dc700 1 -- 172.21.15.143:0/512403789 wait complete. 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5e48dc700 1 Processor -- start 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5e48dc700 1 -- start start 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5e48dc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5cc004810 0x7ff5e0098ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5e48dc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5e00a07a0 0x7ff5e0097670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5e48dc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5e0097bb0 0x7ff5e013a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5e48dc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e009aa90 con 0x7ff5e0097bb0 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5e48dc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e009a600 con 0x7ff5cc004810 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5e48dc700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e00997f0 con 0x7ff5e00a07a0 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5df59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5e0097bb0 0x7ff5e013a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5d7fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5e00a07a0 0x7ff5e0097670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:01.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5df59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5e0097bb0 0x7ff5e013a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:58654/0 (socket says 172.21.15.143:58654) 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5df59e700 1 -- 172.21.15.143:0/1079811288 learned_addr learned my addr 172.21.15.143:0/1079811288 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5ded9d700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5cc004810 0x7ff5e0098ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5df59e700 1 -- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5e00a07a0 msgr2=0x7ff5e0097670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5df59e700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5e00a07a0 0x7ff5e0097670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5df59e700 1 -- 172.21.15.143:0/1079811288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5cc004810 msgr2=0x7ff5e0098ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5df59e700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5cc004810 0x7ff5e0098ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5df59e700 1 -- 172.21.15.143:0/1079811288 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e00981b0 con 0x7ff5e0097bb0 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5df59e700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5e0097bb0 0x7ff5e013a060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00a79b0 tx=0x7ff5d8002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5d6ffd700 1 -- 172.21.15.143:0/1079811288 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d800a950 con 0x7ff5e0097bb0 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5d6ffd700 1 -- 172.21.15.143:0/1079811288 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d800aab0 con 0x7ff5e0097bb0 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5e48dc700 1 -- 172.21.15.143:0/1079811288 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e0098470 con 0x7ff5e0097bb0 2023-09-29T23:33:01.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5d6ffd700 1 -- 172.21.15.143:0/1079811288 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d800ae70 con 0x7ff5e0097bb0 2023-09-29T23:33:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.495+0000 7ff5e48dc700 1 -- 172.21.15.143:0/1079811288 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e013a5a0 con 0x7ff5e0097bb0 2023-09-29T23:33:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.499+0000 7ff5e48dc700 1 -- 172.21.15.143:0/1079811288 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e009a790 con 0x7ff5e0097bb0 2023-09-29T23:33:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.499+0000 7ff5d6ffd700 1 -- 172.21.15.143:0/1079811288 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7ff5d8013600 con 0x7ff5e0097bb0 2023-09-29T23:33:01.502 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.499+0000 7ff5d6ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.499+0000 7ff5d6ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.499+0000 7ff5d6ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.499+0000 7ff5d6ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.499+0000 7ff5d6ffd700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff5c008e8b0 0x7ff5c0090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.499+0000 7ff5ded9d700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff5c008e8b0 0x7ff5c0090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:01.503 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.499+0000 7ff5d6ffd700 1 -- 172.21.15.143:0/1079811288 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7ff5d80b58c0 con 0x7ff5e0097bb0 2023-09-29T23:33:01.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.503+0000 7ff5ded9d700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff5c008e8b0 0x7ff5c0090d60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff5e004ccb0 tx=0x7ff5d0009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:01.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.507+0000 7ff5d6ffd700 1 -- 172.21.15.143:0/1079811288 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7ff5d8086310 con 0x7ff5e0097bb0 2023-09-29T23:33:01.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 -- 172.21.15.143:0/1079811288 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff5e005ecb0 con 0x7ff5e0097bb0 2023-09-29T23:33:01.704 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5d6ffd700 1 -- 172.21.15.143:0/1079811288 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7ff5d8089930 con 0x7ff5e0097bb0 2023-09-29T23:33:01.705 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:01.705 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:01.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 10 mgrc shutdown 2023-09-29T23:33:01.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 -- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff5c008e8b0 msgr2=0x7ff5c0090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:01.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff5c008e8b0 0x7ff5c0090d60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff5e004ccb0 tx=0x7ff5d0009120 comp rx=0 tx=0).stop 2023-09-29T23:33:01.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 -- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5e0097bb0 msgr2=0x7ff5e013a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:01.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5e0097bb0 0x7ff5e013a060 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff5e00a79b0 tx=0x7ff5d8002170 comp rx=0 tx=0).stop 2023-09-29T23:33:01.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 -- 172.21.15.143:0/1079811288 shutdown_connections 2023-09-29T23:33:01.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7ff5c008e8b0 0x7ff5c0090d60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:01.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff5e0097bb0 0x7ff5e013a060 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:01.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7ff5e00a07a0 0x7ff5e0097670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:01.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 --2- 172.21.15.143:0/1079811288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5cc004810 0x7ff5e0098ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:01.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 -- 172.21.15.143:0/1079811288 >> 172.21.15.143:0/1079811288 conn(0x7ff5e00b46e0 msgr2=0x7ff5e013e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:01.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 -- 172.21.15.143:0/1079811288 shutdown_connections 2023-09-29T23:33:01.708 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:01.703+0000 7ff5e48dc700 1 -- 172.21.15.143:0/1079811288 wait complete. 2023-09-29T23:33:04.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:04.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.827+0000 7fc0834a2700 1 Processor -- start 2023-09-29T23:33:04.831 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.827+0000 7fc0834a2700 1 -- start start 2023-09-29T23:33:04.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.827+0000 7fc0834a2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fc07c066ec0 0x7fc07c0a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:04.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.827+0000 7fc0834a2700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc07c09f830 0x7fc07c13d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.827+0000 7fc0834a2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09fc70 0x7fc07c13d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.827+0000 7fc0834a2700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07c13dbc0 con 0x7fc07c066ec0 2023-09-29T23:33:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.827+0000 7fc0834a2700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc07c0a16b0 con 0x7fc07c09f830 2023-09-29T23:33:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.827+0000 7fc0834a2700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c067290 con 0x7fc07c09fc70 2023-09-29T23:33:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc082ca1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09fc70 0x7fc07c13d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc082ca1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09fc70 0x7fc07c13d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:57028/0 (socket says 172.21.15.143:57028) 2023-09-29T23:33:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc082ca1700 1 -- 172.21.15.143:0/3001723243 learned_addr learned my addr 172.21.15.143:0/3001723243 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0824a0700 1 --2- 172.21.15.143:0/3001723243 >> v2:172.21.15.143:3300/0 conn(0x7fc07c066ec0 0x7fc07c0a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc081c9f700 1 --2- 172.21.15.143:0/3001723243 >> v2:172.21.15.156:3300/0 conn(0x7fc07c09f830 0x7fc07c13d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:04.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc082ca1700 1 -- 172.21.15.143:0/3001723243 >> v2:172.21.15.156:3300/0 conn(0x7fc07c09f830 msgr2=0x7fc07c13d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc082ca1700 1 --2- 172.21.15.143:0/3001723243 >> v2:172.21.15.156:3300/0 conn(0x7fc07c09f830 0x7fc07c13d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc082ca1700 1 -- 172.21.15.143:0/3001723243 >> v2:172.21.15.143:3300/0 conn(0x7fc07c066ec0 msgr2=0x7fc07c0a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc082ca1700 1 --2- 172.21.15.143:0/3001723243 >> v2:172.21.15.143:3300/0 conn(0x7fc07c066ec0 0x7fc07c0a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc082ca1700 1 -- 172.21.15.143:0/3001723243 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc07c0a0510 con 0x7fc07c09fc70 2023-09-29T23:33:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0824a0700 1 --2- 172.21.15.143:0/3001723243 >> v2:172.21.15.143:3300/0 conn(0x7fc07c066ec0 0x7fc07c0a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:33:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc082ca1700 1 --2- 172.21.15.143:0/3001723243 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09fc70 0x7fc07c13d680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc07c13ede0 tx=0x7fc074009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b94265f3875fde1b server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc08149e700 1 -- 172.21.15.143:0/3001723243 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07400b860 con 0x7fc07c09fc70 2023-09-29T23:33:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc08149e700 1 -- 172.21.15.143:0/3001723243 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07400b9c0 con 0x7fc07c09fc70 2023-09-29T23:33:04.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc08149e700 1 -- 172.21.15.143:0/3001723243 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07400bd60 con 0x7fc07c09fc70 2023-09-29T23:33:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0834a2700 1 -- 172.21.15.143:0/3001723243 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09fc70 msgr2=0x7fc07c13d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0834a2700 1 --2- 172.21.15.143:0/3001723243 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09fc70 0x7fc07c13d680 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc07c13ede0 tx=0x7fc074009250 comp rx=0 tx=0).stop 2023-09-29T23:33:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0834a2700 1 -- 172.21.15.143:0/3001723243 shutdown_connections 2023-09-29T23:33:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0834a2700 1 --2- 172.21.15.143:0/3001723243 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09fc70 0x7fc07c13d680 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0834a2700 1 --2- 172.21.15.143:0/3001723243 >> v2:172.21.15.156:3300/0 conn(0x7fc07c09f830 0x7fc07c13d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0834a2700 1 --2- 172.21.15.143:0/3001723243 >> v2:172.21.15.143:3300/0 conn(0x7fc07c066ec0 0x7fc07c0a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:04.838 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0834a2700 1 -- 172.21.15.143:0/3001723243 >> 172.21.15.143:0/3001723243 conn(0x7fc07c06f200 msgr2=0x7fc07c069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0834a2700 1 -- 172.21.15.143:0/3001723243 shutdown_connections 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0834a2700 1 -- 172.21.15.143:0/3001723243 wait complete. 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0834a2700 1 Processor -- start 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.831+0000 7fc0834a2700 1 -- start start 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0834a2700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c066ec0 0x7fc07c06b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0834a2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09f830 0x7fc07c099e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0834a2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c09fc70 0x7fc07c1409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0834a2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c06c030 con 0x7fc07c09fc70 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0834a2700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c141b80 con 0x7fc07c066ec0 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0834a2700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07c06bab0 con 0x7fc07c09f830 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc081c9f700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09f830 0x7fc07c099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:04.839 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc081c9f700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09f830 0x7fc07c099e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:57034/0 (socket says 172.21.15.143:57034) 2023-09-29T23:33:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc081c9f700 1 -- 172.21.15.143:0/1885700594 learned_addr learned my addr 172.21.15.143:0/1885700594 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc082ca1700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c09fc70 0x7fc07c1409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0824a0700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c066ec0 0x7fc07c06b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc081c9f700 1 -- 172.21.15.143:0/1885700594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c066ec0 msgr2=0x7fc07c06b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc081c9f700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c066ec0 0x7fc07c06b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc081c9f700 1 -- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c09fc70 msgr2=0x7fc07c1409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc081c9f700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c09fc70 0x7fc07c1409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc081c9f700 1 -- 172.21.15.143:0/1885700594 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0740094f0 con 0x7fc07c09f830 2023-09-29T23:33:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc081c9f700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09f830 0x7fc07c099e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc07800e190 tx=0x7fc078009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0727fc700 1 -- 172.21.15.143:0/1885700594 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800ed20 con 0x7fc07c09f830 2023-09-29T23:33:04.840 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0727fc700 1 -- 172.21.15.143:0/1885700594 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc078014420 con 0x7fc07c09f830 2023-09-29T23:33:04.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0834a2700 1 -- 172.21.15.143:0/1885700594 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07c06bd70 con 0x7fc07c09f830 2023-09-29T23:33:04.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0727fc700 1 -- 172.21.15.143:0/1885700594 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800eea0 con 0x7fc07c09f830 2023-09-29T23:33:04.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0834a2700 1 -- 172.21.15.143:0/1885700594 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07c06c830 con 0x7fc07c09f830 2023-09-29T23:33:04.841 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.835+0000 7fc0834a2700 1 -- 172.21.15.143:0/1885700594 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07c05ecb0 con 0x7fc07c09f830 2023-09-29T23:33:04.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.839+0000 7fc0727fc700 1 -- 172.21.15.143:0/1885700594 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fc0780149f0 con 0x7fc07c09f830 2023-09-29T23:33:04.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.839+0000 7fc0727fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:04.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.839+0000 7fc0727fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:04.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.839+0000 7fc0727fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:04.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.839+0000 7fc0727fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:04.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.839+0000 7fc0727fc700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc06008e8b0 0x7fc060090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:04.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.839+0000 7fc0824a0700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc06008e8b0 0x7fc060090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:04.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.843+0000 7fc0727fc700 1 -- 172.21.15.143:0/1885700594 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fc0780b5ea0 con 0x7fc07c09f830 2023-09-29T23:33:04.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.843+0000 7fc0824a0700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc06008e8b0 0x7fc060090d60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc06c006f00 tx=0x7fc06c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:04.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:04.843+0000 7fc0727fc700 1 -- 172.21.15.143:0/1885700594 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fc0780868f0 con 0x7fc07c09f830 2023-09-29T23:33:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.043+0000 7fc0834a2700 1 -- 172.21.15.143:0/1885700594 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc07c06e400 con 0x7fc07c09f830 2023-09-29T23:33:05.045 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.043+0000 7fc0727fc700 1 -- 172.21.15.143:0/1885700594 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fc078089f10 con 0x7fc07c09f830 2023-09-29T23:33:05.046 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:05.046 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:05.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.043+0000 7fc0834a2700 10 mgrc shutdown 2023-09-29T23:33:05.048 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.043+0000 7fc0834a2700 1 -- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc06008e8b0 msgr2=0x7fc060090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:05.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.043+0000 7fc0834a2700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc06008e8b0 0x7fc060090d60 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc06c006f00 tx=0x7fc06c008040 comp rx=0 tx=0).stop 2023-09-29T23:33:05.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.043+0000 7fc0834a2700 1 -- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09f830 msgr2=0x7fc07c099e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:05.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.043+0000 7fc0834a2700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09f830 0x7fc07c099e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc07800e190 tx=0x7fc078009900 comp rx=0 tx=0).stop 2023-09-29T23:33:05.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.047+0000 7fc0834a2700 1 -- 172.21.15.143:0/1885700594 shutdown_connections 2023-09-29T23:33:05.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.047+0000 7fc0834a2700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fc06008e8b0 0x7fc060090d60 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:05.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.047+0000 7fc0834a2700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc07c09fc70 0x7fc07c1409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:05.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.047+0000 7fc0834a2700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fc07c09f830 0x7fc07c099e50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:05.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.047+0000 7fc0834a2700 1 --2- 172.21.15.143:0/1885700594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc07c066ec0 0x7fc07c06b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:05.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.047+0000 7fc0834a2700 1 -- 172.21.15.143:0/1885700594 >> 172.21.15.143:0/1885700594 conn(0x7fc07c06f200 msgr2=0x7fc07c0a0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:05.049 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.047+0000 7fc0834a2700 1 -- 172.21.15.143:0/1885700594 shutdown_connections 2023-09-29T23:33:05.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:05.047+0000 7fc0834a2700 1 -- 172.21.15.143:0/1885700594 wait complete. 2023-09-29T23:33:08.065 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:08.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.175+0000 7f4907fff700 1 Processor -- start 2023-09-29T23:33:08.179 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.175+0000 7f4907fff700 1 -- start start 2023-09-29T23:33:08.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.175+0000 7f4907fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 0x7f49081435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:08.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.175+0000 7f4907fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4908148920 0x7f490814ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:08.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.175+0000 7f4907fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49080a6980 con 0x7f49080a0e70 2023-09-29T23:33:08.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.175+0000 7f4907fff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f490813ff90 con 0x7f4908148920 2023-09-29T23:33:08.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.175+0000 7f4907fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4908143f70 con 0x7f490806b600 2023-09-29T23:33:08.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.175+0000 7f49067fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 0x7f49081435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:08.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.175+0000 7f49077fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4908148920 0x7f490814ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:08.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.175+0000 7f49067fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 0x7f49081435f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:57038/0 (socket says 172.21.15.143:57038) 2023-09-29T23:33:08.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.175+0000 7f49067fc700 1 -- 172.21.15.143:0/2832830546 learned_addr learned my addr 172.21.15.143:0/2832830546 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:08.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f49067fc700 1 -- 172.21.15.143:0/2832830546 >> v2:172.21.15.143:3300/0 conn(0x7f4908148920 msgr2=0x7f490814ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:08.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f49067fc700 1 --2- 172.21.15.143:0/2832830546 >> v2:172.21.15.143:3300/0 conn(0x7f4908148920 0x7f490814ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:08.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f49067fc700 1 -- 172.21.15.143:0/2832830546 >> v1:172.21.15.156:6789/0 conn(0x7f49080a0e70 legacy=0x7f490806dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:08.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f49067fc700 1 -- 172.21.15.143:0/2832830546 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4908144230 con 0x7f490806b600 2023-09-29T23:33:08.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f49067fc700 1 --2- 172.21.15.143:0/2832830546 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 0x7f49081435f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009f20 tx=0x7f48f8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a40658c2a94ad1b server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:08.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4905ffb700 1 -- 172.21.15.143:0/2832830546 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f800e070 con 0x7f490806b600 2023-09-29T23:33:08.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4905ffb700 1 -- 172.21.15.143:0/2832830546 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f800b9d0 con 0x7f490806b600 2023-09-29T23:33:08.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4905ffb700 1 -- 172.21.15.143:0/2832830546 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f80092e0 con 0x7f490806b600 2023-09-29T23:33:08.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4907fff700 1 -- 172.21.15.143:0/2832830546 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 msgr2=0x7f49081435f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:08.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4907fff700 1 --2- 172.21.15.143:0/2832830546 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 0x7f49081435f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009f20 tx=0x7f48f8009840 comp rx=0 tx=0).stop 2023-09-29T23:33:08.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4907fff700 1 -- 172.21.15.143:0/2832830546 shutdown_connections 2023-09-29T23:33:08.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4907fff700 1 --2- 172.21.15.143:0/2832830546 >> v2:172.21.15.143:3300/0 conn(0x7f4908148920 0x7f490814ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:08.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4907fff700 1 --2- 172.21.15.143:0/2832830546 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 0x7f49081435f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:08.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4907fff700 1 -- 172.21.15.143:0/2832830546 >> 172.21.15.143:0/2832830546 conn(0x7f49080b4640 msgr2=0x7f49080b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:08.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4907fff700 1 -- 172.21.15.143:0/2832830546 shutdown_connections 2023-09-29T23:33:08.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4907fff700 1 -- 172.21.15.143:0/2832830546 wait complete. 2023-09-29T23:33:08.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4907fff700 1 Processor -- start 2023-09-29T23:33:08.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.179+0000 7f4907fff700 1 -- start start 2023-09-29T23:33:08.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f4907fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 0x7f490813dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:08.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f4907fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49080a0e70 0x7f49081461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:08.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f4907fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4908148920 0x7f4908150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:08.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f4907fff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4908150120 con 0x7f49080a0e70 2023-09-29T23:33:08.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f4907fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490813d800 con 0x7f4908148920 2023-09-29T23:33:08.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f4907fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4908150350 con 0x7f490806b600 2023-09-29T23:33:08.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f49067fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49080a0e70 0x7f49081461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:08.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f49067fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49080a0e70 0x7f49081461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59016/0 (socket says 172.21.15.143:59016) 2023-09-29T23:33:08.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f49067fc700 1 -- 172.21.15.143:0/968354598 learned_addr learned my addr 172.21.15.143:0/968354598 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:08.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f4906ffd700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 0x7f490813dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:08.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f49077fe700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4908148920 0x7f4908150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f49067fc700 1 -- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 msgr2=0x7f490813dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f49067fc700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 0x7f490813dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f49067fc700 1 -- 172.21.15.143:0/968354598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4908148920 msgr2=0x7f4908150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f49067fc700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4908148920 0x7f4908150780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f49067fc700 1 -- 172.21.15.143:0/968354598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f8009ae0 con 0x7f49080a0e70 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f49077fe700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4908148920 0x7f4908150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f49067fc700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49080a0e70 0x7f49081461c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f48f800b610 tx=0x7f48f8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f48eeffd700 1 -- 172.21.15.143:0/968354598 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f80089a0 con 0x7f49080a0e70 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f48eeffd700 1 -- 172.21.15.143:0/968354598 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f8008b00 con 0x7f49080a0e70 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f4907fff700 1 -- 172.21.15.143:0/968354598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490813e320 con 0x7f49080a0e70 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f48eeffd700 1 -- 172.21.15.143:0/968354598 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8026460 con 0x7f49080a0e70 2023-09-29T23:33:08.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f4907fff700 1 -- 172.21.15.143:0/968354598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490813c7e0 con 0x7f49080a0e70 2023-09-29T23:33:08.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.183+0000 7f4907fff700 1 -- 172.21.15.143:0/968354598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490805ecb0 con 0x7f49080a0e70 2023-09-29T23:33:08.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.187+0000 7f48eeffd700 1 -- 172.21.15.143:0/968354598 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f48f8003040 con 0x7f49080a0e70 2023-09-29T23:33:08.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.187+0000 7f48eeffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:08.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.187+0000 7f48eeffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:08.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.187+0000 7f48eeffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:08.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.187+0000 7f48eeffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:08.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.187+0000 7f48eeffd700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f48f408e8b0 0x7f48f4090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:08.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.187+0000 7f4906ffd700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f48f408e8b0 0x7f48f4090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:08.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.187+0000 7f48eeffd700 1 -- 172.21.15.143:0/968354598 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f48f80b64e0 con 0x7f49080a0e70 2023-09-29T23:33:08.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.187+0000 7f4906ffd700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f48f408e8b0 0x7f48f4090d60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0039f0 tx=0x7f48fc007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:08.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.199+0000 7f48eeffd700 1 -- 172.21.15.143:0/968354598 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f48f8086f30 con 0x7f49080a0e70 2023-09-29T23:33:08.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.395+0000 7f4907fff700 1 -- 172.21.15.143:0/968354598 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4908144450 con 0x7f49080a0e70 2023-09-29T23:33:08.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.395+0000 7f48eeffd700 1 -- 172.21.15.143:0/968354598 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f48f808a550 con 0x7f49080a0e70 2023-09-29T23:33:08.400 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:08.400 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:08.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 10 mgrc shutdown 2023-09-29T23:33:08.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 -- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f48f408e8b0 msgr2=0x7f48f4090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:08.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f48f408e8b0 0x7f48f4090d60 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0039f0 tx=0x7f48fc007ec0 comp rx=0 tx=0).stop 2023-09-29T23:33:08.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 -- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49080a0e70 msgr2=0x7f49081461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:08.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49080a0e70 0x7f49081461c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f48f800b610 tx=0x7f48f8018000 comp rx=0 tx=0).stop 2023-09-29T23:33:08.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 -- 172.21.15.143:0/968354598 shutdown_connections 2023-09-29T23:33:08.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f48f408e8b0 0x7f48f4090d60 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:08.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4908148920 0x7f4908150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:08.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49080a0e70 0x7f49081461c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:08.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 --2- 172.21.15.143:0/968354598 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f490806b600 0x7f490813dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:08.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 -- 172.21.15.143:0/968354598 >> 172.21.15.143:0/968354598 conn(0x7f49080b4640 msgr2=0x7f49081422c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:08.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 -- 172.21.15.143:0/968354598 shutdown_connections 2023-09-29T23:33:08.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:08.399+0000 7f4907fff700 1 -- 172.21.15.143:0/968354598 wait complete. 2023-09-29T23:33:11.415 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:11.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.523+0000 7fd4767df700 1 Processor -- start 2023-09-29T23:33:11.527 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.523+0000 7fd4767df700 1 -- start start 2023-09-29T23:33:11.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a01c0 0x7fd4700a0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:11.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd47009e2c0 0x7fd47009e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:11.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47009f480 con 0x7fd4700a4260 2023-09-29T23:33:11.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4700a2cc0 con 0x7fd47009e2c0 2023-09-29T23:33:11.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700a1080 con 0x7fd4700a01c0 2023-09-29T23:33:11.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4757dd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a01c0 0x7fd4700a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:11.530 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4757dd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a01c0 0x7fd4700a0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:57046/0 (socket says 172.21.15.143:57046) 2023-09-29T23:33:11.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4757dd700 1 -- 172.21.15.143:0/26388674 learned_addr learned my addr 172.21.15.143:0/26388674 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:11.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd474fdc700 1 --2- 172.21.15.143:0/26388674 >> v2:172.21.15.156:3300/0 conn(0x7fd47009e2c0 0x7fd47009e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:11.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4757dd700 1 -- 172.21.15.143:0/26388674 >> v2:172.21.15.156:3300/0 conn(0x7fd47009e2c0 msgr2=0x7fd47009e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:11.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4757dd700 1 --2- 172.21.15.143:0/26388674 >> v2:172.21.15.156:3300/0 conn(0x7fd47009e2c0 0x7fd47009e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:11.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4757dd700 1 -- 172.21.15.143:0/26388674 >> v1:172.21.15.156:6789/0 conn(0x7fd4700a4260 legacy=0x7fd4700a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:11.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4757dd700 1 -- 172.21.15.143:0/26388674 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47009ef30 con 0x7fd4700a01c0 2023-09-29T23:33:11.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd474fdc700 1 --2- 172.21.15.143:0/26388674 >> v2:172.21.15.156:3300/0 conn(0x7fd47009e2c0 0x7fd47009e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-29T23:33:11.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4757dd700 1 --2- 172.21.15.143:0/26388674 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a01c0 0x7fd4700a0590 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00ab50 tx=0x7fd46c00a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e6f431f7ba8366 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:11.531 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd467fff700 1 -- 172.21.15.143:0/26388674 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c00ca20 con 0x7fd4700a01c0 2023-09-29T23:33:11.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd467fff700 1 -- 172.21.15.143:0/26388674 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c00cb80 con 0x7fd4700a01c0 2023-09-29T23:33:11.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd467fff700 1 -- 172.21.15.143:0/26388674 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c0183f0 con 0x7fd4700a01c0 2023-09-29T23:33:11.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 -- 172.21.15.143:0/26388674 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a01c0 msgr2=0x7fd4700a0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:11.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 --2- 172.21.15.143:0/26388674 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a01c0 0x7fd4700a0590 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00ab50 tx=0x7fd46c00a3f0 comp rx=0 tx=0).stop 2023-09-29T23:33:11.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 -- 172.21.15.143:0/26388674 shutdown_connections 2023-09-29T23:33:11.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 --2- 172.21.15.143:0/26388674 >> v2:172.21.15.156:3300/0 conn(0x7fd47009e2c0 0x7fd47009e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:11.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 --2- 172.21.15.143:0/26388674 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a01c0 0x7fd4700a0590 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:11.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 -- 172.21.15.143:0/26388674 >> 172.21.15.143:0/26388674 conn(0x7fd4700b2ca0 msgr2=0x7fd4700b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:11.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.527+0000 7fd4767df700 1 -- 172.21.15.143:0/26388674 shutdown_connections 2023-09-29T23:33:11.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4767df700 1 -- 172.21.15.143:0/26388674 wait complete. 2023-09-29T23:33:11.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4767df700 1 Processor -- start 2023-09-29T23:33:11.534 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4767df700 1 -- start start 2023-09-29T23:33:11.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4767df700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd47009e2c0 0x7fd47013d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:11.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4767df700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4700a01c0 0x7fd470151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:11.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4767df700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a4260 0x7fd470151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:11.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4767df700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47014b520 con 0x7fd47009e2c0 2023-09-29T23:33:11.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4767df700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd470151ca0 con 0x7fd4700a01c0 2023-09-29T23:33:11.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4767df700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd470099870 con 0x7fd4700a4260 2023-09-29T23:33:11.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4757dd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd47009e2c0 0x7fd47013d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:11.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd475fde700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a4260 0x7fd470151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:11.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4757dd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd47009e2c0 0x7fd47013d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59026/0 (socket says 172.21.15.143:59026) 2023-09-29T23:33:11.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4757dd700 1 -- 172.21.15.143:0/2118358976 learned_addr learned my addr 172.21.15.143:0/2118358976 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:11.535 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd474fdc700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4700a01c0 0x7fd470151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:11.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4757dd700 1 -- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a4260 msgr2=0x7fd470151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:11.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4757dd700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a4260 0x7fd470151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:11.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4757dd700 1 -- 172.21.15.143:0/2118358976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4700a01c0 msgr2=0x7fd470151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:11.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4757dd700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4700a01c0 0x7fd470151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:11.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4757dd700 1 -- 172.21.15.143:0/2118358976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c00a690 con 0x7fd47009e2c0 2023-09-29T23:33:11.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd475fde700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a4260 0x7fd470151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:33:11.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4757dd700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd47009e2c0 0x7fd47013d490 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00a430 tx=0x7fd46c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:11.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4657fa700 1 -- 172.21.15.143:0/2118358976 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c010600 con 0x7fd47009e2c0 2023-09-29T23:33:11.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.531+0000 7fd4657fa700 1 -- 172.21.15.143:0/2118358976 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c010760 con 0x7fd47009e2c0 2023-09-29T23:33:11.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.535+0000 7fd4767df700 1 -- 172.21.15.143:0/2118358976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd470098820 con 0x7fd47009e2c0 2023-09-29T23:33:11.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.535+0000 7fd4657fa700 1 -- 172.21.15.143:0/2118358976 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c010b20 con 0x7fd47009e2c0 2023-09-29T23:33:11.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.535+0000 7fd4767df700 1 -- 172.21.15.143:0/2118358976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd470095a40 con 0x7fd47009e2c0 2023-09-29T23:33:11.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.535+0000 7fd4767df700 1 -- 172.21.15.143:0/2118358976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4700989b0 con 0x7fd47009e2c0 2023-09-29T23:33:11.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.535+0000 7fd4657fa700 1 -- 172.21.15.143:0/2118358976 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fd46c02a3b0 con 0x7fd47009e2c0 2023-09-29T23:33:11.539 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.535+0000 7fd4657fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:11.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.535+0000 7fd4657fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:11.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.535+0000 7fd4657fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:11.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.535+0000 7fd4657fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:11.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.535+0000 7fd4657fa700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd45408e8b0 0x7fd454090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:11.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.539+0000 7fd474fdc700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd45408e8b0 0x7fd454090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:11.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.539+0000 7fd4657fa700 1 -- 172.21.15.143:0/2118358976 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fd46c0b5f90 con 0x7fd47009e2c0 2023-09-29T23:33:11.540 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.539+0000 7fd474fdc700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd45408e8b0 0x7fd454090d60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd4700676f0 tx=0x7fd460006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:11.546 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.543+0000 7fd4657fa700 1 -- 172.21.15.143:0/2118358976 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd46c0869e0 con 0x7fd47009e2c0 2023-09-29T23:33:11.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.739+0000 7fd4767df700 1 -- 172.21.15.143:0/2118358976 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd47005ecb0 con 0x7fd47009e2c0 2023-09-29T23:33:11.743 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.739+0000 7fd4657fa700 1 -- 172.21.15.143:0/2118358976 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fd46c08a000 con 0x7fd47009e2c0 2023-09-29T23:33:11.743 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:11.743 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:11.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 10 mgrc shutdown 2023-09-29T23:33:11.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 -- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd45408e8b0 msgr2=0x7fd454090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:11.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd45408e8b0 0x7fd454090d60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd4700676f0 tx=0x7fd460006f90 comp rx=0 tx=0).stop 2023-09-29T23:33:11.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 -- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd47009e2c0 msgr2=0x7fd47013d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:11.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd47009e2c0 0x7fd47013d490 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd46c00a430 tx=0x7fd46c019000 comp rx=0 tx=0).stop 2023-09-29T23:33:11.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 -- 172.21.15.143:0/2118358976 shutdown_connections 2023-09-29T23:33:11.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd45408e8b0 0x7fd454090d60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:11.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd4700a4260 0x7fd470151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:11.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4700a01c0 0x7fd470151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:11.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 --2- 172.21.15.143:0/2118358976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd47009e2c0 0x7fd47013d490 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:11.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 -- 172.21.15.143:0/2118358976 >> 172.21.15.143:0/2118358976 conn(0x7fd4700b2ca0 msgr2=0x7fd47006edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:11.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 -- 172.21.15.143:0/2118358976 shutdown_connections 2023-09-29T23:33:11.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:11.743+0000 7fd4767df700 1 -- 172.21.15.143:0/2118358976 wait complete. 2023-09-29T23:33:14.764 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83aac65700 1 Processor -- start 2023-09-29T23:33:14.894 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83aac65700 1 -- start start 2023-09-29T23:33:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83aac65700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f83a406f060 0x7f83a40a45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83aac65700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a4143db0 0x7f83a414aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83aac65700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a40a4b30 con 0x7f83a40a3570 2023-09-29T23:33:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83aac65700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83a40a62e0 con 0x7f83a406f060 2023-09-29T23:33:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83aac65700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a406f430 con 0x7f83a4143db0 2023-09-29T23:33:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83a9202700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a4143db0 0x7f83a414aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83a9202700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a4143db0 0x7f83a414aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:35168/0 (socket says 172.21.15.143:35168) 2023-09-29T23:33:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83a9202700 1 -- 172.21.15.143:0/1913352678 learned_addr learned my addr 172.21.15.143:0/1913352678 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:14.895 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83a8a01700 1 --2- 172.21.15.143:0/1913352678 >> v2:172.21.15.156:3300/0 conn(0x7f83a406f060 0x7f83a40a45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:14.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83a9202700 1 -- 172.21.15.143:0/1913352678 >> v2:172.21.15.156:3300/0 conn(0x7f83a406f060 msgr2=0x7f83a40a45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:14.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83a9202700 1 --2- 172.21.15.143:0/1913352678 >> v2:172.21.15.156:3300/0 conn(0x7f83a406f060 0x7f83a40a45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:14.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83a9202700 1 -- 172.21.15.143:0/1913352678 >> v1:172.21.15.156:6789/0 conn(0x7f83a40a3570 legacy=0x7f83a41436a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:14.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83a9202700 1 -- 172.21.15.143:0/1913352678 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a41408e0 con 0x7f83a4143db0 2023-09-29T23:33:14.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.891+0000 7f83a9202700 1 --2- 172.21.15.143:0/1913352678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a4143db0 0x7f83a414aa40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f839c009f20 tx=0x7f839c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c28d61ef15fef423 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:14.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83a37fe700 1 -- 172.21.15.143:0/1913352678 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c00e070 con 0x7f83a4143db0 2023-09-29T23:33:14.897 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83a37fe700 1 -- 172.21.15.143:0/1913352678 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839c00b9d0 con 0x7f83a4143db0 2023-09-29T23:33:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83a37fe700 1 -- 172.21.15.143:0/1913352678 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839c0092e0 con 0x7f83a4143db0 2023-09-29T23:33:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 -- 172.21.15.143:0/1913352678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a4143db0 msgr2=0x7f83a414aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 --2- 172.21.15.143:0/1913352678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a4143db0 0x7f83a414aa40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f839c009f20 tx=0x7f839c009840 comp rx=0 tx=0).stop 2023-09-29T23:33:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 -- 172.21.15.143:0/1913352678 shutdown_connections 2023-09-29T23:33:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 --2- 172.21.15.143:0/1913352678 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a4143db0 0x7f83a414aa40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 --2- 172.21.15.143:0/1913352678 >> v2:172.21.15.156:3300/0 conn(0x7f83a406f060 0x7f83a40a45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 -- 172.21.15.143:0/1913352678 >> 172.21.15.143:0/1913352678 conn(0x7f83a4067720 msgr2=0x7f83a40b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 -- 172.21.15.143:0/1913352678 shutdown_connections 2023-09-29T23:33:14.898 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 -- 172.21.15.143:0/1913352678 wait complete. 2023-09-29T23:33:14.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 Processor -- start 2023-09-29T23:33:14.899 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 -- start start 2023-09-29T23:33:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a406f060 0x7f83a4153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83a40a3570 0x7f83a409e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83a4143db0 0x7f83a409e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a4153fe0 con 0x7f83a4143db0 2023-09-29T23:33:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a4144760 con 0x7f83a40a3570 2023-09-29T23:33:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83aac65700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a4144a20 con 0x7f83a406f060 2023-09-29T23:33:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83a8a01700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a406f060 0x7f83a4153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83a8a01700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a406f060 0x7f83a4153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:35176/0 (socket says 172.21.15.143:35176) 2023-09-29T23:33:14.901 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83a8a01700 1 -- 172.21.15.143:0/3148433716 learned_addr learned my addr 172.21.15.143:0/3148433716 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83a9202700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83a4143db0 0x7f83a409e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.895+0000 7f83a3fff700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83a40a3570 0x7f83a409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83a8a01700 1 -- 172.21.15.143:0/3148433716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83a40a3570 msgr2=0x7f83a409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83a8a01700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83a40a3570 0x7f83a409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83a8a01700 1 -- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83a4143db0 msgr2=0x7f83a409e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83a8a01700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83a4143db0 0x7f83a409e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83a8a01700 1 -- 172.21.15.143:0/3148433716 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c009ae0 con 0x7f83a406f060 2023-09-29T23:33:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83a8a01700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a406f060 0x7f83a4153610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f839800ec80 tx=0x7f83980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83a0ff9700 1 -- 172.21.15.143:0/3148433716 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800c850 con 0x7f83a406f060 2023-09-29T23:33:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83a0ff9700 1 -- 172.21.15.143:0/3148433716 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f839800c9d0 con 0x7f83a406f060 2023-09-29T23:33:14.902 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83aac65700 1 -- 172.21.15.143:0/3148433716 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a40a23f0 con 0x7f83a406f060 2023-09-29T23:33:14.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83a0ff9700 1 -- 172.21.15.143:0/3148433716 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800cd90 con 0x7f83a406f060 2023-09-29T23:33:14.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83aac65700 1 -- 172.21.15.143:0/3148433716 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a4153b50 con 0x7f83a406f060 2023-09-29T23:33:14.903 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.899+0000 7f83aac65700 1 -- 172.21.15.143:0/3148433716 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a4153da0 con 0x7f83a406f060 2023-09-29T23:33:14.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.903+0000 7f83a0ff9700 1 -- 172.21.15.143:0/3148433716 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f8398013680 con 0x7f83a406f060 2023-09-29T23:33:14.905 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.903+0000 7f83a0ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:14.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.903+0000 7f83a0ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:14.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.903+0000 7f83a0ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:14.906 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.903+0000 7f83a0ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:14.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.903+0000 7f83a0ff9700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f839008e8b0 0x7f8390090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:14.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.903+0000 7f83a3fff700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f839008e8b0 0x7f8390090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:14.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.903+0000 7f83a0ff9700 1 -- 172.21.15.143:0/3148433716 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f83980b5080 con 0x7f83a406f060 2023-09-29T23:33:14.907 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.903+0000 7f83a3fff700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f839008e8b0 0x7f8390090d60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f83a41468a0 tx=0x7f8394000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:14.910 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:14.907+0000 7f83a0ff9700 1 -- 172.21.15.143:0/3148433716 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8398085ad0 con 0x7f83a406f060 2023-09-29T23:33:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 -- 172.21.15.143:0/3148433716 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f83a4153da0 con 0x7f83a406f060 2023-09-29T23:33:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83a0ff9700 1 -- 172.21.15.143:0/3148433716 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f8398017100 con 0x7f83a406f060 2023-09-29T23:33:15.105 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:15.105 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:15.107 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 10 mgrc shutdown 2023-09-29T23:33:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 -- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f839008e8b0 msgr2=0x7f8390090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f839008e8b0 0x7f8390090d60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f83a41468a0 tx=0x7f8394000bc0 comp rx=0 tx=0).stop 2023-09-29T23:33:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 -- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a406f060 msgr2=0x7f83a4153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a406f060 0x7f83a4153610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f839800ec80 tx=0x7f83980091a0 comp rx=0 tx=0).stop 2023-09-29T23:33:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 -- 172.21.15.143:0/3148433716 shutdown_connections 2023-09-29T23:33:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f839008e8b0 0x7f8390090d60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f83a4143db0 0x7f83a409e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83a40a3570 0x7f83a409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 --2- 172.21.15.143:0/3148433716 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f83a406f060 0x7f83a4153610 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 -- 172.21.15.143:0/3148433716 >> 172.21.15.143:0/3148433716 conn(0x7f83a4067720 msgr2=0x7f83a40684f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 -- 172.21.15.143:0/3148433716 shutdown_connections 2023-09-29T23:33:15.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:15.103+0000 7f83aac65700 1 -- 172.21.15.143:0/3148433716 wait complete. 2023-09-29T23:33:18.120 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:18.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b863fe700 1 Processor -- start 2023-09-29T23:33:18.234 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b863fe700 1 -- start start 2023-09-29T23:33:18.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b863fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b80066ec0 0x7f8b8006c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:18.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b863fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b8006e930 0x7f8b800a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:18.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b863fe700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8006cd10 con 0x7f8b8013c010 2023-09-29T23:33:18.235 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b863fe700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b80067290 con 0x7f8b8006e930 2023-09-29T23:33:18.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b863fe700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8006ad80 con 0x7f8b80066ec0 2023-09-29T23:33:18.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b853fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b80066ec0 0x7f8b8006c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:18.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b84bfb700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f8b8006e930 0x7f8b800a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:18.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b853fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b80066ec0 0x7f8b8006c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:35182/0 (socket says 172.21.15.143:35182) 2023-09-29T23:33:18.236 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b853fc700 1 -- 172.21.15.143:0/1605658324 learned_addr learned my addr 172.21.15.143:0/1605658324 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:18.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b84bfb700 1 -- 172.21.15.143:0/1605658324 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b80066ec0 msgr2=0x7f8b8006c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:18.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b84bfb700 1 --2- 172.21.15.143:0/1605658324 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b80066ec0 0x7f8b8006c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:18.237 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.231+0000 7f8b84bfb700 1 -- 172.21.15.143:0/1605658324 >> v1:172.21.15.156:6789/0 conn(0x7f8b8013c010 legacy=0x7f8b8013c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:18.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b84bfb700 1 -- 172.21.15.143:0/1605658324 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8006b040 con 0x7f8b8006e930 2023-09-29T23:33:18.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b853fc700 1 --2- 172.21.15.143:0/1605658324 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b80066ec0 0x7f8b8006c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:33:18.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b84bfb700 1 --2- 172.21.15.143:0/1605658324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8006e930 0x7f8b800a1e40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8b700098e0 tx=0x7f8b700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbb4eec7d8a9335 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:18.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b77fff700 1 -- 172.21.15.143:0/1605658324 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7000b640 con 0x7f8b8006e930 2023-09-29T23:33:18.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b77fff700 1 -- 172.21.15.143:0/1605658324 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7000b7a0 con 0x7f8b8006e930 2023-09-29T23:33:18.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b77fff700 1 -- 172.21.15.143:0/1605658324 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7000bb40 con 0x7f8b8006e930 2023-09-29T23:33:18.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b863fe700 1 -- 172.21.15.143:0/1605658324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8006e930 msgr2=0x7f8b800a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:18.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b863fe700 1 --2- 172.21.15.143:0/1605658324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8006e930 0x7f8b800a1e40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8b700098e0 tx=0x7f8b700091a0 comp rx=0 tx=0).stop 2023-09-29T23:33:18.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b863fe700 1 -- 172.21.15.143:0/1605658324 shutdown_connections 2023-09-29T23:33:18.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b863fe700 1 --2- 172.21.15.143:0/1605658324 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8006e930 0x7f8b800a1e40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:18.238 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b863fe700 1 --2- 172.21.15.143:0/1605658324 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b80066ec0 0x7f8b8006c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:18.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b863fe700 1 -- 172.21.15.143:0/1605658324 >> 172.21.15.143:0/1605658324 conn(0x7f8b8006f200 msgr2=0x7f8b80069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:18.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b863fe700 1 -- 172.21.15.143:0/1605658324 shutdown_connections 2023-09-29T23:33:18.239 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.235+0000 7f8b863fe700 1 -- 172.21.15.143:0/1605658324 wait complete. 2023-09-29T23:33:18.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b863fe700 1 Processor -- start 2023-09-29T23:33:18.240 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b863fe700 1 -- start start 2023-09-29T23:33:18.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b863fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b80066ec0 0x7f8b80140960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:18.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b863fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8006e930 0x7f8b8009a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:18.242 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b863fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b8013c010 0x7f8b801501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b863fe700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8009ea00 con 0x7f8b8006e930 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b863fe700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b801414e0 con 0x7f8b80066ec0 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b863fe700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8009ab80 con 0x7f8b8013c010 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b84bfb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8006e930 0x7f8b8009a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b85bfd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b8013c010 0x7f8b801501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b84bfb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8006e930 0x7f8b8009a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54776/0 (socket says 172.21.15.143:54776) 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b84bfb700 1 -- 172.21.15.143:0/3618732923 learned_addr learned my addr 172.21.15.143:0/3618732923 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b853fc700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b80066ec0 0x7f8b80140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b85bfd700 1 -- 172.21.15.143:0/3618732923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b80066ec0 msgr2=0x7f8b80140960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b85bfd700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b80066ec0 0x7f8b80140960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b85bfd700 1 -- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8006e930 msgr2=0x7f8b8009a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:18.243 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b85bfd700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8006e930 0x7f8b8009a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:18.244 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b85bfd700 1 -- 172.21.15.143:0/3618732923 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b7c0094f0 con 0x7f8b8013c010 2023-09-29T23:33:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b85bfd700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b8013c010 0x7f8b801501b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c7c0 tx=0x7f8b780015f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b757fa700 1 -- 172.21.15.143:0/3618732923 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78011590 con 0x7f8b8013c010 2023-09-29T23:33:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b757fa700 1 -- 172.21.15.143:0/3618732923 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78011710 con 0x7f8b8013c010 2023-09-29T23:33:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b757fa700 1 -- 172.21.15.143:0/3618732923 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78011ad0 con 0x7f8b8013c010 2023-09-29T23:33:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b863fe700 1 -- 172.21.15.143:0/3618732923 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b70009420 con 0x7f8b8013c010 2023-09-29T23:33:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.239+0000 7f8b863fe700 1 -- 172.21.15.143:0/3618732923 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b80096b40 con 0x7f8b8013c010 2023-09-29T23:33:18.245 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.243+0000 7f8b863fe700 1 -- 172.21.15.143:0/3618732923 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8005ecb0 con 0x7f8b8013c010 2023-09-29T23:33:18.247 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.243+0000 7f8b757fa700 1 -- 172.21.15.143:0/3618732923 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f8b78003030 con 0x7f8b8013c010 2023-09-29T23:33:18.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.243+0000 7f8b757fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:18.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.247+0000 7f8b757fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:18.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.247+0000 7f8b757fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:18.248 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.247+0000 7f8b757fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.247+0000 7f8b757fa700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8b6408e8e0 0x7f8b64090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.247+0000 7f8b757fa700 1 -- 172.21.15.143:0/3618732923 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f8b780b41f0 con 0x7f8b8013c010 2023-09-29T23:33:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.247+0000 7f8b853fc700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8b6408e8e0 0x7f8b64090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:18.249 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.247+0000 7f8b853fc700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8b6408e8e0 0x7f8b64090d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c006d20 tx=0x7f8b7c009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:18.255 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.251+0000 7f8b757fa700 1 -- 172.21.15.143:0/3618732923 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8b78084c40 con 0x7f8b8013c010 2023-09-29T23:33:18.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.451+0000 7f8b863fe700 1 -- 172.21.15.143:0/3618732923 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8b8006b840 con 0x7f8b8013c010 2023-09-29T23:33:18.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.451+0000 7f8b757fa700 1 -- 172.21.15.143:0/3618732923 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f8b78088260 con 0x7f8b8013c010 2023-09-29T23:33:18.456 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:18.456 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:18.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.455+0000 7f8b863fe700 10 mgrc shutdown 2023-09-29T23:33:18.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.455+0000 7f8b863fe700 1 -- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8b6408e8e0 msgr2=0x7f8b64090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:18.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.455+0000 7f8b863fe700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8b6408e8e0 0x7f8b64090d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c006d20 tx=0x7f8b7c009250 comp rx=0 tx=0).stop 2023-09-29T23:33:18.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.455+0000 7f8b863fe700 1 -- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b8013c010 msgr2=0x7f8b801501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:18.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.455+0000 7f8b863fe700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b8013c010 0x7f8b801501b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800c7c0 tx=0x7f8b780015f0 comp rx=0 tx=0).stop 2023-09-29T23:33:18.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.455+0000 7f8b863fe700 1 -- 172.21.15.143:0/3618732923 shutdown_connections 2023-09-29T23:33:18.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.455+0000 7f8b863fe700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8b6408e8e0 0x7f8b64090d90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:18.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.455+0000 7f8b863fe700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8b8013c010 0x7f8b801501b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:18.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.455+0000 7f8b863fe700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8b8006e930 0x7f8b8009a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:18.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.455+0000 7f8b863fe700 1 --2- 172.21.15.143:0/3618732923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b80066ec0 0x7f8b80140960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:18.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.455+0000 7f8b863fe700 1 -- 172.21.15.143:0/3618732923 >> 172.21.15.143:0/3618732923 conn(0x7f8b8006f200 msgr2=0x7f8b8013b300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:18.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.459+0000 7f8b863fe700 1 -- 172.21.15.143:0/3618732923 shutdown_connections 2023-09-29T23:33:18.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:18.459+0000 7f8b863fe700 1 -- 172.21.15.143:0/3618732923 wait complete. 2023-09-29T23:33:21.478 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:21.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbffa332700 1 Processor -- start 2023-09-29T23:33:21.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbffa332700 1 -- start start 2023-09-29T23:33:21.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbffa332700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbff413fce0 0x7fbff40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:21.593 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbffa332700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff40a48e0 con 0x7fbff40b71e0 2023-09-29T23:33:21.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbffa332700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff40a3de0 con 0x7fbff40a5310 2023-09-29T23:33:21.594 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbffa332700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff40a40a0 con 0x7fbff413fce0 2023-09-29T23:33:21.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff9b31700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fbff40b71e0 0x7fbff414aa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:46250/0 (socket says 172.21.15.143:46250) 2023-09-29T23:33:21.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff9b31700 1 -- 172.21.15.143:0/2413041276 learned_addr learned my addr 172.21.15.143:0/2413041276 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:33:21.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff8b2f700 1 --2- 172.21.15.143:0/2413041276 >> v2:172.21.15.156:3300/0 conn(0x7fbff413fce0 0x7fbff40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:21.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff3fff700 1 -- 172.21.15.143:0/2413041276 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591252392 0 0) 0x7fbff40a48e0 con 0x7fbff40b71e0 2023-09-29T23:33:21.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff3fff700 1 -- 172.21.15.143:0/2413041276 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfdc003570 con 0x7fbff40b71e0 2023-09-29T23:33:21.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff8b2f700 1 -- 172.21.15.143:0/2413041276 >> v1:172.21.15.156:6789/0 conn(0x7fbff40a5310 legacy=0x7fbff413f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:21.596 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff8b2f700 1 -- 172.21.15.143:0/2413041276 >> v1:172.21.15.143:6789/0 conn(0x7fbff40b71e0 legacy=0x7fbff414aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:21.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff3fff700 1 -- 172.21.15.143:0/2413041276 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2635000663 0 0) 0x7fbfdc003570 con 0x7fbff40b71e0 2023-09-29T23:33:21.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff8b2f700 1 -- 172.21.15.143:0/2413041276 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff40b67e0 con 0x7fbff413fce0 2023-09-29T23:33:21.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff8b2f700 1 --2- 172.21.15.143:0/2413041276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff413fce0 0x7fbff40a3770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009e60 tx=0x7fbfe4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc069b557b691965 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:21.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff3fff700 1 -- 172.21.15.143:0/2413041276 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe400e070 con 0x7fbff413fce0 2023-09-29T23:33:21.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff3fff700 1 -- 172.21.15.143:0/2413041276 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe400b850 con 0x7fbff413fce0 2023-09-29T23:33:21.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbff3fff700 1 -- 172.21.15.143:0/2413041276 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec001490 con 0x7fbff413fce0 2023-09-29T23:33:21.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbffa332700 1 -- 172.21.15.143:0/2413041276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff413fce0 msgr2=0x7fbff40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:21.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbffa332700 1 --2- 172.21.15.143:0/2413041276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff413fce0 0x7fbff40a3770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009e60 tx=0x7fbfe4009780 comp rx=0 tx=0).stop 2023-09-29T23:33:21.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbffa332700 1 -- 172.21.15.143:0/2413041276 shutdown_connections 2023-09-29T23:33:21.597 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbffa332700 1 --2- 172.21.15.143:0/2413041276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff413fce0 0x7fbff40a3770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:21.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.591+0000 7fbffa332700 1 -- 172.21.15.143:0/2413041276 >> 172.21.15.143:0/2413041276 conn(0x7fbff4066c90 msgr2=0x7fbff4067090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:21.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbffa332700 1 -- 172.21.15.143:0/2413041276 shutdown_connections 2023-09-29T23:33:21.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbffa332700 1 -- 172.21.15.143:0/2413041276 wait complete. 2023-09-29T23:33:21.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbffa332700 1 Processor -- start 2023-09-29T23:33:21.598 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbffa332700 1 -- start start 2023-09-29T23:33:21.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbffa332700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbff40a5310 0x7fbff40a29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:21.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbffa332700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbff40b71e0 0x7fbff409b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:21.600 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbffa332700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff413fce0 0x7fbff406e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:21.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbffa332700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff409a3d0 con 0x7fbff40b71e0 2023-09-29T23:33:21.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbffa332700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff409ba20 con 0x7fbff413fce0 2023-09-29T23:33:21.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbffa332700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff4153d90 con 0x7fbff40a5310 2023-09-29T23:33:21.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbff8b2f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbff40b71e0 0x7fbff409b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:21.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbff8b2f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbff40b71e0 0x7fbff409b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54784/0 (socket says 172.21.15.143:54784) 2023-09-29T23:33:21.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbff8b2f700 1 -- 172.21.15.143:0/2766521938 learned_addr learned my addr 172.21.15.143:0/2766521938 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:21.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbff9330700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbff40a5310 0x7fbff40a29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:21.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbff9b31700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff413fce0 0x7fbff406e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:21.601 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbff8b2f700 1 -- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbff40a5310 msgr2=0x7fbff40a29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbff8b2f700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbff40a5310 0x7fbff40a29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.595+0000 7fbff8b2f700 1 -- 172.21.15.143:0/2766521938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff413fce0 msgr2=0x7fbff406e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.599+0000 7fbff8b2f700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff413fce0 0x7fbff406e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.599+0000 7fbff8b2f700 1 -- 172.21.15.143:0/2766521938 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe4009a20 con 0x7fbff40b71e0 2023-09-29T23:33:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.599+0000 7fbff8b2f700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbff40b71e0 0x7fbff409b4e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40099f0 tx=0x7fbfe4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.599+0000 7fbff17fa700 1 -- 172.21.15.143:0/2766521938 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4012a30 con 0x7fbff40b71e0 2023-09-29T23:33:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.599+0000 7fbff17fa700 1 -- 172.21.15.143:0/2766521938 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe4012b90 con 0x7fbff40b71e0 2023-09-29T23:33:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.599+0000 7fbffa332700 1 -- 172.21.15.143:0/2766521938 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff41534b0 con 0x7fbff40b71e0 2023-09-29T23:33:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.599+0000 7fbff17fa700 1 -- 172.21.15.143:0/2766521938 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe401e450 con 0x7fbff40b71e0 2023-09-29T23:33:21.602 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.599+0000 7fbffa332700 1 -- 172.21.15.143:0/2766521938 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff406e920 con 0x7fbff40b71e0 2023-09-29T23:33:21.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.599+0000 7fbffa332700 1 -- 172.21.15.143:0/2766521938 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff409b200 con 0x7fbff40b71e0 2023-09-29T23:33:21.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.599+0000 7fbff17fa700 1 -- 172.21.15.143:0/2766521938 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fbfe401e7c0 con 0x7fbff40b71e0 2023-09-29T23:33:21.603 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.599+0000 7fbff17fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:21.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.603+0000 7fbff17fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:21.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.603+0000 7fbff17fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:21.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.603+0000 7fbff17fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:21.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.603+0000 7fbff17fa700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fbfe008e8b0 0x7fbfe0090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:21.605 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.603+0000 7fbff9330700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fbfe008e8b0 0x7fbfe0090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:21.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.603+0000 7fbff17fa700 1 -- 172.21.15.143:0/2766521938 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fbfe40b6650 con 0x7fbff40b71e0 2023-09-29T23:33:21.606 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.603+0000 7fbff9330700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fbfe008e8b0 0x7fbfe0090d60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8003a80 tx=0x7fbfe8007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:21.611 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.607+0000 7fbff17fa700 1 -- 172.21.15.143:0/2766521938 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fbff409b200 con 0x7fbff40b71e0 2023-09-29T23:33:21.808 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 -- 172.21.15.143:0/2766521938 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fbff409b200 con 0x7fbff40b71e0 2023-09-29T23:33:21.809 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbff17fa700 1 -- 172.21.15.143:0/2766521938 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fbff409b200 con 0x7fbff40b71e0 2023-09-29T23:33:21.809 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:21.809 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:21.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 10 mgrc shutdown 2023-09-29T23:33:21.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 -- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fbfe008e8b0 msgr2=0x7fbfe0090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:21.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fbfe008e8b0 0x7fbfe0090d60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8003a80 tx=0x7fbfe8007ec0 comp rx=0 tx=0).stop 2023-09-29T23:33:21.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 -- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbff40b71e0 msgr2=0x7fbff409b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:21.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbff40b71e0 0x7fbff409b4e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbfe40099f0 tx=0x7fbfe4017000 comp rx=0 tx=0).stop 2023-09-29T23:33:21.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 -- 172.21.15.143:0/2766521938 shutdown_connections 2023-09-29T23:33:21.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fbfe008e8b0 0x7fbfe0090d60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:21.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbff413fce0 0x7fbff406e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:21.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbff40b71e0 0x7fbff409b4e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:21.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 --2- 172.21.15.143:0/2766521938 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fbff40a5310 0x7fbff40a29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:21.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 -- 172.21.15.143:0/2766521938 >> 172.21.15.143:0/2766521938 conn(0x7fbff4066c90 msgr2=0x7fbff40b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:21.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 -- 172.21.15.143:0/2766521938 shutdown_connections 2023-09-29T23:33:21.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:21.807+0000 7fbffa332700 1 -- 172.21.15.143:0/2766521938 wait complete. 2023-09-29T23:33:24.823 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:24.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 Processor -- start 2023-09-29T23:33:24.936 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 -- start start 2023-09-29T23:33:24.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9978066ec0 0x7f997809fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:24.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f997806b680 0x7f99780a0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:24.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9978067290 con 0x7f997809f450 2023-09-29T23:33:24.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99780a0b40 con 0x7f997806b680 2023-09-29T23:33:24.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9978067ad0 con 0x7f9978066ec0 2023-09-29T23:33:24.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997d8bc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f997806b680 0x7f99780a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:24.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997d8bc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f997806b680 0x7f99780a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56512/0 (socket says 172.21.15.143:56512) 2023-09-29T23:33:24.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997d8bc700 1 -- 172.21.15.143:0/4091654920 learned_addr learned my addr 172.21.15.143:0/4091654920 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:24.938 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997e8be700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f997809f450 0x7f9978135e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:45960/0 (socket says 172.21.15.143:45960) 2023-09-29T23:33:24.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997e0bd700 1 --2- 172.21.15.143:0/4091654920 >> v2:172.21.15.156:3300/0 conn(0x7f9978066ec0 0x7f997809fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:24.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997d8bc700 1 -- 172.21.15.143:0/4091654920 >> v2:172.21.15.156:3300/0 conn(0x7f9978066ec0 msgr2=0x7f997809fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:24.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997d8bc700 1 --2- 172.21.15.143:0/4091654920 >> v2:172.21.15.156:3300/0 conn(0x7f9978066ec0 0x7f997809fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:24.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997d8bc700 1 -- 172.21.15.143:0/4091654920 >> v1:172.21.15.143:6789/0 conn(0x7f997809f450 legacy=0x7f9978135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:24.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997d8bc700 1 -- 172.21.15.143:0/4091654920 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99780a0dc0 con 0x7f997806b680 2023-09-29T23:33:24.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997e0bd700 1 --2- 172.21.15.143:0/4091654920 >> v2:172.21.15.156:3300/0 conn(0x7f9978066ec0 0x7f997809fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:33:24.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997d8bc700 1 --2- 172.21.15.143:0/4091654920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997806b680 0x7f99780a0190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f99780a0fb0 tx=0x7f9974009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1b030cbc5feedc server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:24.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997d0bb700 1 -- 172.21.15.143:0/4091654920 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997400b830 con 0x7f997806b680 2023-09-29T23:33:24.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997d0bb700 1 -- 172.21.15.143:0/4091654920 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997400b990 con 0x7f997806b680 2023-09-29T23:33:24.939 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997d0bb700 1 -- 172.21.15.143:0/4091654920 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997400bd50 con 0x7f997806b680 2023-09-29T23:33:24.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 -- 172.21.15.143:0/4091654920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997806b680 msgr2=0x7f99780a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:24.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 --2- 172.21.15.143:0/4091654920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997806b680 0x7f99780a0190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f99780a0fb0 tx=0x7f9974009250 comp rx=0 tx=0).stop 2023-09-29T23:33:24.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 -- 172.21.15.143:0/4091654920 shutdown_connections 2023-09-29T23:33:24.940 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 --2- 172.21.15.143:0/4091654920 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f997806b680 0x7f99780a0190 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:24.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 --2- 172.21.15.143:0/4091654920 >> v2:172.21.15.156:3300/0 conn(0x7f9978066ec0 0x7f997809fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:24.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 -- 172.21.15.143:0/4091654920 >> 172.21.15.143:0/4091654920 conn(0x7f997806f200 msgr2=0x7f9978069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:24.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 -- 172.21.15.143:0/4091654920 shutdown_connections 2023-09-29T23:33:24.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.935+0000 7f997f0bf700 1 -- 172.21.15.143:0/4091654920 wait complete. 2023-09-29T23:33:24.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997f0bf700 1 Processor -- start 2023-09-29T23:33:24.941 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997f0bf700 1 -- start start 2023-09-29T23:33:24.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997f0bf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9978066ec0 0x7f9978137a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:24.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997f0bf700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f997806b680 0x7f99780b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:24.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997f0bf700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f997809f450 0x7f997809be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:24.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997f0bf700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9978150600 con 0x7f9978066ec0 2023-09-29T23:33:24.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997f0bf700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99781508c0 con 0x7f997806b680 2023-09-29T23:33:24.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997f0bf700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9978150b80 con 0x7f997809f450 2023-09-29T23:33:24.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997e0bd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9978066ec0 0x7f9978137a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:24.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997e8be700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f997809f450 0x7f997809be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:24.944 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997e0bd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9978066ec0 0x7f9978137a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56524/0 (socket says 172.21.15.143:56524) 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997e0bd700 1 -- 172.21.15.143:0/3920949445 learned_addr learned my addr 172.21.15.143:0/3920949445 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997d8bc700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f997806b680 0x7f99780b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997e8be700 1 -- 172.21.15.143:0/3920949445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f997806b680 msgr2=0x7f99780b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997e8be700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f997806b680 0x7f99780b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997e8be700 1 -- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9978066ec0 msgr2=0x7f9978137a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997e8be700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9978066ec0 0x7f9978137a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997e8be700 1 -- 172.21.15.143:0/3920949445 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740094f0 con 0x7f997809f450 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997e8be700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f997809f450 0x7f997809be30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f997000cba0 tx=0x7f99700019b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f996e7fc700 1 -- 172.21.15.143:0/3920949445 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970005df0 con 0x7f997809f450 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f996e7fc700 1 -- 172.21.15.143:0/3920949445 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997000a5d0 con 0x7f997809f450 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.939+0000 7f997f0bf700 1 -- 172.21.15.143:0/3920949445 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997809eff0 con 0x7f997809f450 2023-09-29T23:33:24.945 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.943+0000 7f996e7fc700 1 -- 172.21.15.143:0/3920949445 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000a950 con 0x7f997809f450 2023-09-29T23:33:24.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.943+0000 7f997f0bf700 1 -- 172.21.15.143:0/3920949445 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9978138380 con 0x7f997809f450 2023-09-29T23:33:24.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.943+0000 7f997f0bf700 1 -- 172.21.15.143:0/3920949445 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9978138050 con 0x7f997809f450 2023-09-29T23:33:24.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.943+0000 7f996e7fc700 1 -- 172.21.15.143:0/3920949445 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f997000ad00 con 0x7f997809f450 2023-09-29T23:33:24.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.943+0000 7f996e7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:24.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.947+0000 7f996e7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:24.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.947+0000 7f996e7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:24.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.947+0000 7f996e7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:24.948 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.947+0000 7f996e7fc700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f995c08e8e0 0x7f995c090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:24.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.947+0000 7f997e0bd700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f995c08e8e0 0x7f995c090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:24.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.947+0000 7f996e7fc700 1 -- 172.21.15.143:0/3920949445 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f99700b46b0 con 0x7f997809f450 2023-09-29T23:33:24.949 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.947+0000 7f997e0bd700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f995c08e8e0 0x7f995c090d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9968006dc0 tx=0x7f9968008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:24.955 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:24.951+0000 7f996e7fc700 1 -- 172.21.15.143:0/3920949445 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f9970085100 con 0x7f997809f450 2023-09-29T23:33:25.154 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.151+0000 7f997f0bf700 1 -- 172.21.15.143:0/3920949445 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f997805ecb0 con 0x7f997809f450 2023-09-29T23:33:25.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.151+0000 7f996e7fc700 1 -- 172.21.15.143:0/3920949445 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f9970088720 con 0x7f997809f450 2023-09-29T23:33:25.155 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:25.155 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:25.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 10 mgrc shutdown 2023-09-29T23:33:25.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 -- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f995c08e8e0 msgr2=0x7f995c090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:25.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f995c08e8e0 0x7f995c090d90 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9968006dc0 tx=0x7f9968008040 comp rx=0 tx=0).stop 2023-09-29T23:33:25.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 -- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f997809f450 msgr2=0x7f997809be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:25.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f997809f450 0x7f997809be30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f997000cba0 tx=0x7f99700019b0 comp rx=0 tx=0).stop 2023-09-29T23:33:25.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 -- 172.21.15.143:0/3920949445 shutdown_connections 2023-09-29T23:33:25.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f995c08e8e0 0x7f995c090d90 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:25.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f997809f450 0x7f997809be30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:25.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f997806b680 0x7f99780b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:25.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 --2- 172.21.15.143:0/3920949445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9978066ec0 0x7f9978137a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:25.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 -- 172.21.15.143:0/3920949445 >> 172.21.15.143:0/3920949445 conn(0x7f997806f200 msgr2=0x7f997806dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:25.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 -- 172.21.15.143:0/3920949445 shutdown_connections 2023-09-29T23:33:25.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:25.155+0000 7f997f0bf700 1 -- 172.21.15.143:0/3920949445 wait complete. 2023-09-29T23:33:26.267 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:33:26.270 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:33:28.169 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:28.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0359e700 1 Processor -- start 2023-09-29T23:33:28.282 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0359e700 1 -- start start 2023-09-29T23:33:28.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0359e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d04132640 0x7f4d0406ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:28.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0359e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d0409f550 0x7f4d0413f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:28.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0359e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d0406c240 con 0x7f4d0409f960 2023-09-29T23:33:28.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0359e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d040a0460 con 0x7f4d04132640 2023-09-29T23:33:28.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0359e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0409fee0 con 0x7f4d0409f550 2023-09-29T23:33:28.283 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0259c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d04132640 0x7f4d0406ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:28.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0259c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f4d04132640 0x7f4d0406ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56534/0 (socket says 172.21.15.143:56534) 2023-09-29T23:33:28.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d01d9b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d0409f550 0x7f4d0413f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:28.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0259c700 1 -- 172.21.15.143:0/4034233801 learned_addr learned my addr 172.21.15.143:0/4034233801 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:28.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0259c700 1 -- 172.21.15.143:0/4034233801 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d0409f550 msgr2=0x7f4d0413f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:28.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0259c700 1 --2- 172.21.15.143:0/4034233801 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d0409f550 0x7f4d0413f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:28.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0259c700 1 -- 172.21.15.143:0/4034233801 >> v1:172.21.15.156:6789/0 conn(0x7f4d0409f960 legacy=0x7f4d0413fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:28.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0259c700 1 -- 172.21.15.143:0/4034233801 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d040a01a0 con 0x7f4d04132640 2023-09-29T23:33:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.279+0000 7f4d0259c700 1 --2- 172.21.15.143:0/4034233801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d04132640 0x7f4d0406ba00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009f20 tx=0x7f4cfc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b335ed6267e43b server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0159a700 1 -- 172.21.15.143:0/4034233801 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00e070 con 0x7f4d04132640 2023-09-29T23:33:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0159a700 1 -- 172.21.15.143:0/4034233801 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc00b9d0 con 0x7f4d04132640 2023-09-29T23:33:28.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0159a700 1 -- 172.21.15.143:0/4034233801 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc0092e0 con 0x7f4d04132640 2023-09-29T23:33:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0359e700 1 -- 172.21.15.143:0/4034233801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d04132640 msgr2=0x7f4d0406ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0359e700 1 --2- 172.21.15.143:0/4034233801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d04132640 0x7f4d0406ba00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009f20 tx=0x7f4cfc009840 comp rx=0 tx=0).stop 2023-09-29T23:33:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0359e700 1 -- 172.21.15.143:0/4034233801 shutdown_connections 2023-09-29T23:33:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0359e700 1 --2- 172.21.15.143:0/4034233801 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d0409f550 0x7f4d0413f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0359e700 1 --2- 172.21.15.143:0/4034233801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d04132640 0x7f4d0406ba00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0359e700 1 -- 172.21.15.143:0/4034233801 >> 172.21.15.143:0/4034233801 conn(0x7f4d0406f200 msgr2=0x7f4d04069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0359e700 1 -- 172.21.15.143:0/4034233801 shutdown_connections 2023-09-29T23:33:28.286 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0359e700 1 -- 172.21.15.143:0/4034233801 wait complete. 2023-09-29T23:33:28.287 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0359e700 1 Processor -- start 2023-09-29T23:33:28.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.283+0000 7f4d0359e700 1 -- start start 2023-09-29T23:33:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d0359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0409f550 0x7f4d0413fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d0359e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d0409f960 0x7f4d040a9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d0359e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d04132640 0x7f4d040aa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d0359e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040a9440 con 0x7f4d0409f960 2023-09-29T23:33:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d0359e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040aa5c0 con 0x7f4d0409f550 2023-09-29T23:33:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d0359e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d040a74c0 con 0x7f4d04132640 2023-09-29T23:33:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d01d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d0409f960 0x7f4d040a9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d02d9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d04132640 0x7f4d040aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d01d9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d0409f960 0x7f4d040a9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56536/0 (socket says 172.21.15.143:56536) 2023-09-29T23:33:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d01d9b700 1 -- 172.21.15.143:0/1081232070 learned_addr learned my addr 172.21.15.143:0/1081232070 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:28.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d0259c700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0409f550 0x7f4d0413fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:28.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d01d9b700 1 -- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d04132640 msgr2=0x7f4d040aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:28.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d01d9b700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d04132640 0x7f4d040aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:28.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d01d9b700 1 -- 172.21.15.143:0/1081232070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0409f550 msgr2=0x7f4d0413fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:28.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d01d9b700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0409f550 0x7f4d0413fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:28.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d01d9b700 1 -- 172.21.15.143:0/1081232070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc009ae0 con 0x7f4d0409f960 2023-09-29T23:33:28.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d01d9b700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d0409f960 0x7f4d040a9b40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800e930 tx=0x7f4cf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:28.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4cf27fc700 1 -- 172.21.15.143:0/1081232070 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8013680 con 0x7f4d0409f960 2023-09-29T23:33:28.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4cf27fc700 1 -- 172.21.15.143:0/1081232070 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf80137e0 con 0x7f4d0409f960 2023-09-29T23:33:28.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d0359e700 1 -- 172.21.15.143:0/1081232070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d040a7710 con 0x7f4d0409f960 2023-09-29T23:33:28.291 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4cf27fc700 1 -- 172.21.15.143:0/1081232070 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf8013b80 con 0x7f4d0409f960 2023-09-29T23:33:28.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.287+0000 7f4d0359e700 1 -- 172.21.15.143:0/1081232070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0413b010 con 0x7f4d0409f960 2023-09-29T23:33:28.292 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.291+0000 7f4d0359e700 1 -- 172.21.15.143:0/1081232070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0405ecb0 con 0x7f4d0409f960 2023-09-29T23:33:28.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.291+0000 7f4cf27fc700 1 -- 172.21.15.143:0/1081232070 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f4cf800c3e0 con 0x7f4d0409f960 2023-09-29T23:33:28.294 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.291+0000 7f4cf27fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:28.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.291+0000 7f4cf27fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:28.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.291+0000 7f4cf27fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:28.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.291+0000 7f4cf27fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:28.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.291+0000 7f4cf27fc700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4ce408e830 0x7f4ce4090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:28.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.291+0000 7f4cf27fc700 1 -- 172.21.15.143:0/1081232070 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f4cf80b48f0 con 0x7f4d0409f960 2023-09-29T23:33:28.295 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.291+0000 7f4d0259c700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4ce408e830 0x7f4ce4090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:28.296 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.295+0000 7f4d0259c700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4ce408e830 0x7f4ce4090ce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009880 tx=0x7f4cfc018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:28.305 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.303+0000 7f4cf27fc700 1 -- 172.21.15.143:0/1081232070 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4cf8085340 con 0x7f4d0409f960 2023-09-29T23:33:28.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.499+0000 7f4d0359e700 1 -- 172.21.15.143:0/1081232070 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4d040a7200 con 0x7f4d0409f960 2023-09-29T23:33:28.504 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4cf27fc700 1 -- 172.21.15.143:0/1081232070 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f4cf8088960 con 0x7f4d0409f960 2023-09-29T23:33:28.504 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:28.504 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:28.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 10 mgrc shutdown 2023-09-29T23:33:28.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 -- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4ce408e830 msgr2=0x7f4ce4090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:28.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4ce408e830 0x7f4ce4090ce0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009880 tx=0x7f4cfc018000 comp rx=0 tx=0).stop 2023-09-29T23:33:28.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 -- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d0409f960 msgr2=0x7f4d040a9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d0409f960 0x7f4d040a9b40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4cf800e930 tx=0x7f4cf80091a0 comp rx=0 tx=0).stop 2023-09-29T23:33:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 -- 172.21.15.143:0/1081232070 shutdown_connections 2023-09-29T23:33:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4ce408e830 0x7f4ce4090ce0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4d04132640 0x7f4d040aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4d0409f960 0x7f4d040a9b40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 --2- 172.21.15.143:0/1081232070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d0409f550 0x7f4d0413fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 -- 172.21.15.143:0/1081232070 >> 172.21.15.143:0/1081232070 conn(0x7f4d0406f200 msgr2=0x7f4d040b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 -- 172.21.15.143:0/1081232070 shutdown_connections 2023-09-29T23:33:28.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:28.503+0000 7f4d0359e700 1 -- 172.21.15.143:0/1081232070 wait complete. 2023-09-29T23:33:31.518 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:31.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.623+0000 7f2cda14a700 1 Processor -- start 2023-09-29T23:33:31.628 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 -- start start 2023-09-29T23:33:31.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2cd4066ec0 0x7f2cd40a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:31.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2cd409fc70 0x7f2cd4148590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:31.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd40a16b0 con 0x7f2cd409f830 2023-09-29T23:33:31.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd4067290 con 0x7f2cd409fc70 2023-09-29T23:33:31.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd40a0510 con 0x7f2cd4066ec0 2023-09-29T23:33:31.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cd9949700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f2cd409fc70 0x7f2cd4148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:31.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cd8947700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f2cd409f830 0x7f2cd413d140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.143:44660/0 (socket says 172.21.15.143:44660) 2023-09-29T23:33:31.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cd8947700 1 -- 172.21.15.143:0/4176159027 learned_addr learned my addr 172.21.15.143:0/4176159027 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:33:31.629 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cd9148700 1 --2- 172.21.15.143:0/4176159027 >> v2:172.21.15.156:3300/0 conn(0x7f2cd4066ec0 0x7f2cd40a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2ccbfff700 1 -- 172.21.15.143:0/4176159027 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992806271 0 0) 0x7f2cd40a16b0 con 0x7f2cd409f830 2023-09-29T23:33:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2ccbfff700 1 -- 172.21.15.143:0/4176159027 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cc0003570 con 0x7f2cd409f830 2023-09-29T23:33:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cd9949700 1 -- 172.21.15.143:0/4176159027 >> v2:172.21.15.156:3300/0 conn(0x7f2cd4066ec0 msgr2=0x7f2cd40a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cd9949700 1 --2- 172.21.15.143:0/4176159027 >> v2:172.21.15.156:3300/0 conn(0x7f2cd4066ec0 0x7f2cd40a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cd9949700 1 -- 172.21.15.143:0/4176159027 >> v1:172.21.15.156:6789/0 conn(0x7f2cd409f830 legacy=0x7f2cd413d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cd9949700 1 -- 172.21.15.143:0/4176159027 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd406e400 con 0x7f2cd409fc70 2023-09-29T23:33:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cd9949700 1 --2- 172.21.15.143:0/4176159027 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd409fc70 0x7f2cd4148590 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40a4fb0 tx=0x7f2ccc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac6c9bd36b308de server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2ccbfff700 1 -- 172.21.15.143:0/4176159027 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc00e070 con 0x7f2cd409fc70 2023-09-29T23:33:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2ccbfff700 1 -- 172.21.15.143:0/4176159027 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc00bab0 con 0x7f2cd409fc70 2023-09-29T23:33:31.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2ccbfff700 1 -- 172.21.15.143:0/4176159027 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc00be30 con 0x7f2cd409fc70 2023-09-29T23:33:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 -- 172.21.15.143:0/4176159027 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd409fc70 msgr2=0x7f2cd4148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:31.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 --2- 172.21.15.143:0/4176159027 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd409fc70 0x7f2cd4148590 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40a4fb0 tx=0x7f2ccc009780 comp rx=0 tx=0).stop 2023-09-29T23:33:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 -- 172.21.15.143:0/4176159027 shutdown_connections 2023-09-29T23:33:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 --2- 172.21.15.143:0/4176159027 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd409fc70 0x7f2cd4148590 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 --2- 172.21.15.143:0/4176159027 >> v2:172.21.15.156:3300/0 conn(0x7f2cd4066ec0 0x7f2cd40a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 -- 172.21.15.143:0/4176159027 >> 172.21.15.143:0/4176159027 conn(0x7f2cd406f200 msgr2=0x7f2cd4069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 -- 172.21.15.143:0/4176159027 shutdown_connections 2023-09-29T23:33:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.627+0000 7f2cda14a700 1 -- 172.21.15.143:0/4176159027 wait complete. 2023-09-29T23:33:31.633 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cda14a700 1 Processor -- start 2023-09-29T23:33:31.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cda14a700 1 -- start start 2023-09-29T23:33:31.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cda14a700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2cd4066ec0 0x7f2cd4144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cda14a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cd409f830 0x7f2cd4137e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cda14a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd409fc70 0x7f2cd4138370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cda14a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4143c60 con 0x7f2cd409fc70 2023-09-29T23:33:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cda14a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4144180 con 0x7f2cd409f830 2023-09-29T23:33:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cda14a700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd41439a0 con 0x7f2cd4066ec0 2023-09-29T23:33:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9148700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2cd4066ec0 0x7f2cd4144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9949700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd409fc70 0x7f2cd4138370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9148700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2cd4066ec0 0x7f2cd4144b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:39430/0 (socket says 172.21.15.143:39430) 2023-09-29T23:33:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd8947700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cd409f830 0x7f2cd4137e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9949700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd409fc70 0x7f2cd4138370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:56546/0 (socket says 172.21.15.143:56546) 2023-09-29T23:33:31.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9148700 1 -- 172.21.15.143:0/3961665404 learned_addr learned my addr 172.21.15.143:0/3961665404 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:31.636 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9148700 1 -- 172.21.15.143:0/3961665404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cd409f830 msgr2=0x7f2cd4137e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9148700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cd409f830 0x7f2cd4137e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9148700 1 -- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd409fc70 msgr2=0x7f2cd4138370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9148700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd409fc70 0x7f2cd4138370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9148700 1 -- 172.21.15.143:0/3961665404 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc009a20 con 0x7f2cd4066ec0 2023-09-29T23:33:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9949700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd409fc70 0x7f2cd4138370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:33:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.631+0000 7f2cd9148700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2cd4066ec0 0x7f2cd4144b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400ec80 tx=0x7f2cc40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cc97fa700 1 -- 172.21.15.143:0/3961665404 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400c850 con 0x7f2cd4066ec0 2023-09-29T23:33:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cc97fa700 1 -- 172.21.15.143:0/3961665404 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc400c9b0 con 0x7f2cd4066ec0 2023-09-29T23:33:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cda14a700 1 -- 172.21.15.143:0/3961665404 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd406d9e0 con 0x7f2cd4066ec0 2023-09-29T23:33:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cc97fa700 1 -- 172.21.15.143:0/3961665404 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400cd50 con 0x7f2cd4066ec0 2023-09-29T23:33:31.637 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cda14a700 1 -- 172.21.15.143:0/3961665404 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd406cbc0 con 0x7f2cd4066ec0 2023-09-29T23:33:31.638 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cda14a700 1 -- 172.21.15.143:0/3961665404 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd40a1e50 con 0x7f2cd4066ec0 2023-09-29T23:33:31.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cc97fa700 1 -- 172.21.15.143:0/3961665404 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f2cc4013680 con 0x7f2cd4066ec0 2023-09-29T23:33:31.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cc97fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:31.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cc97fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:31.639 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cc97fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:31.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cc97fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:31.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.635+0000 7f2cc97fa700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2cb808e830 0x7f2cb8090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:31.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.639+0000 7f2cd8947700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2cb808e830 0x7f2cb8090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:31.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.639+0000 7f2cc97fa700 1 -- 172.21.15.143:0/3961665404 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f2cc40b4a80 con 0x7f2cd4066ec0 2023-09-29T23:33:31.640 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.639+0000 7f2cd8947700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2cb808e830 0x7f2cb8090ce0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0007fd0 tx=0x7f2cd0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:31.649 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.647+0000 7f2cc97fa700 1 -- 172.21.15.143:0/3961665404 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2cd40a1e50 con 0x7f2cd4066ec0 2023-09-29T23:33:31.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.843+0000 7f2cda14a700 1 -- 172.21.15.143:0/3961665404 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2cd4064e80 con 0x7f2cd4066ec0 2023-09-29T23:33:31.845 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.843+0000 7f2cc97fa700 1 -- 172.21.15.143:0/3961665404 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f2cc4017100 con 0x7f2cd4066ec0 2023-09-29T23:33:31.846 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:31.846 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:31.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.843+0000 7f2cda14a700 10 mgrc shutdown 2023-09-29T23:33:31.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.843+0000 7f2cda14a700 1 -- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2cb808e830 msgr2=0x7f2cb8090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:31.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.843+0000 7f2cda14a700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2cb808e830 0x7f2cb8090ce0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0007fd0 tx=0x7f2cd0009040 comp rx=0 tx=0).stop 2023-09-29T23:33:31.848 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.843+0000 7f2cda14a700 1 -- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2cd4066ec0 msgr2=0x7f2cd4144b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:31.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.847+0000 7f2cda14a700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2cd4066ec0 0x7f2cd4144b20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400ec80 tx=0x7f2cc40091a0 comp rx=0 tx=0).stop 2023-09-29T23:33:31.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.847+0000 7f2cda14a700 1 -- 172.21.15.143:0/3961665404 shutdown_connections 2023-09-29T23:33:31.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.847+0000 7f2cda14a700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2cb808e830 0x7f2cb8090ce0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:31.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.847+0000 7f2cda14a700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2cd409fc70 0x7f2cd4138370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:31.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.847+0000 7f2cda14a700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cd409f830 0x7f2cd4137e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:31.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.847+0000 7f2cda14a700 1 --2- 172.21.15.143:0/3961665404 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2cd4066ec0 0x7f2cd4144b20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:31.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.847+0000 7f2cda14a700 1 -- 172.21.15.143:0/3961665404 >> 172.21.15.143:0/3961665404 conn(0x7f2cd406f200 msgr2=0x7f2cd413b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:31.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.847+0000 7f2cda14a700 1 -- 172.21.15.143:0/3961665404 shutdown_connections 2023-09-29T23:33:31.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:31.847+0000 7f2cda14a700 1 -- 172.21.15.143:0/3961665404 wait complete. 2023-09-29T23:33:34.861 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:34.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9f59e700 1 Processor -- start 2023-09-29T23:33:34.973 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9f59e700 1 -- start start 2023-09-29T23:33:34.974 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9f59e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8aa0066ec0 0x7f8aa009fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9f59e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 0x7f8aa00a0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9f59e700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa0067290 con 0x7f8aa006b680 2023-09-29T23:33:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9f59e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa00a0b40 con 0x7f8aa0066ec0 2023-09-29T23:33:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9f59e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0067ad0 con 0x7f8aa0136520 2023-09-29T23:33:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9ed9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 0x7f8aa00a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9ed9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 0x7f8aa00a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:39178/0 (socket says 172.21.15.143:39178) 2023-09-29T23:33:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9e59c700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8aa0066ec0 0x7f8aa009fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9ed9d700 1 -- 172.21.15.143:0/1454630500 learned_addr learned my addr 172.21.15.143:0/1454630500 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:34.975 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9dd9b700 1 --1- 172.21.15.143:0/1454630500 >> v1:172.21.15.156:6789/0 conn(0x7f8aa006b680 0x7f8aa0135e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.143:44532/0 (socket says 172.21.15.143:44532) 2023-09-29T23:33:34.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9d59a700 1 -- 172.21.15.143:0/1454630500 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2073158193 0 0) 0x7f8aa0067290 con 0x7f8aa006b680 2023-09-29T23:33:34.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9ed9d700 1 -- 172.21.15.143:0/1454630500 >> v2:172.21.15.156:3300/0 conn(0x7f8aa0066ec0 msgr2=0x7f8aa009fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:34.977 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9ed9d700 1 --2- 172.21.15.143:0/1454630500 >> v2:172.21.15.156:3300/0 conn(0x7f8aa0066ec0 0x7f8aa009fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:34.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9ed9d700 1 -- 172.21.15.143:0/1454630500 >> v1:172.21.15.156:6789/0 conn(0x7f8aa006b680 legacy=0x7f8aa0135e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:34.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9ed9d700 1 -- 172.21.15.143:0/1454630500 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00a0dc0 con 0x7f8aa0136520 2023-09-29T23:33:34.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.971+0000 7f8a9ed9d700 1 --2- 172.21.15.143:0/1454630500 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 0x7f8aa00a0190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009f20 tx=0x7f8a98009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea3c2a3a7f29dad server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:34.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9d59a700 1 -- 172.21.15.143:0/1454630500 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800e070 con 0x7f8aa0136520 2023-09-29T23:33:34.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9d59a700 1 -- 172.21.15.143:0/1454630500 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9800b9d0 con 0x7f8aa0136520 2023-09-29T23:33:34.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9d59a700 1 -- 172.21.15.143:0/1454630500 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a980092e0 con 0x7f8aa0136520 2023-09-29T23:33:34.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 -- 172.21.15.143:0/1454630500 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 msgr2=0x7f8aa00a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:34.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 --2- 172.21.15.143:0/1454630500 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 0x7f8aa00a0190 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009f20 tx=0x7f8a98009840 comp rx=0 tx=0).stop 2023-09-29T23:33:34.978 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 -- 172.21.15.143:0/1454630500 shutdown_connections 2023-09-29T23:33:34.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 --2- 172.21.15.143:0/1454630500 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 0x7f8aa00a0190 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:34.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 --2- 172.21.15.143:0/1454630500 >> v2:172.21.15.156:3300/0 conn(0x7f8aa0066ec0 0x7f8aa009fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:34.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 -- 172.21.15.143:0/1454630500 >> 172.21.15.143:0/1454630500 conn(0x7f8aa006f200 msgr2=0x7f8aa0069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:34.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 -- 172.21.15.143:0/1454630500 shutdown_connections 2023-09-29T23:33:34.979 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 -- 172.21.15.143:0/1454630500 wait complete. 2023-09-29T23:33:34.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 Processor -- start 2023-09-29T23:33:34.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 -- start start 2023-09-29T23:33:34.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aa0066ec0 0x7f8aa0141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:34.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa006b680 0x7f8aa0132790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:34.981 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 0x7f8aa00b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:34.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0131040 con 0x7f8aa006b680 2023-09-29T23:33:34.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0141690 con 0x7f8aa0066ec0 2023-09-29T23:33:34.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9f59e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa0142080 con 0x7f8aa0136520 2023-09-29T23:33:34.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9dd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa006b680 0x7f8aa0132790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:34.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9ed9d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 0x7f8aa00b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:34.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9dd9b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa006b680 0x7f8aa0132790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54142/0 (socket says 172.21.15.143:54142) 2023-09-29T23:33:34.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.975+0000 7f8a9dd9b700 1 -- 172.21.15.143:0/2252168700 learned_addr learned my addr 172.21.15.143:0/2252168700 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:34.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a9e59c700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aa0066ec0 0x7f8aa0141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:34.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a9dd9b700 1 -- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 msgr2=0x7f8aa00b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:34.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a9dd9b700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 0x7f8aa00b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:34.982 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a9dd9b700 1 -- 172.21.15.143:0/2252168700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aa0066ec0 msgr2=0x7f8aa0141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:34.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a9dd9b700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aa0066ec0 0x7f8aa0141b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:34.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a9dd9b700 1 -- 172.21.15.143:0/2252168700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a98009ae0 con 0x7f8aa006b680 2023-09-29T23:33:34.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a9dd9b700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa006b680 0x7f8aa0132790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000c7e0 tx=0x7f8a90007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:34.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a8e7fc700 1 -- 172.21.15.143:0/2252168700 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90005a70 con 0x7f8aa006b680 2023-09-29T23:33:34.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a8e7fc700 1 -- 172.21.15.143:0/2252168700 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a90005bd0 con 0x7f8aa006b680 2023-09-29T23:33:34.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a9f59e700 1 -- 172.21.15.143:0/2252168700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa01428a0 con 0x7f8aa006b680 2023-09-29T23:33:34.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a8e7fc700 1 -- 172.21.15.143:0/2252168700 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a900070d0 con 0x7f8aa006b680 2023-09-29T23:33:34.983 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a9f59e700 1 -- 172.21.15.143:0/2252168700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa0132cd0 con 0x7f8aa006b680 2023-09-29T23:33:34.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.979+0000 7f8a9f59e700 1 -- 172.21.15.143:0/2252168700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa005ecb0 con 0x7f8aa006b680 2023-09-29T23:33:34.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.983+0000 7f8a8e7fc700 1 -- 172.21.15.143:0/2252168700 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f8a90007440 con 0x7f8aa006b680 2023-09-29T23:33:34.984 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.983+0000 7f8a8e7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:34.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.983+0000 7f8a8e7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:34.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.983+0000 7f8a8e7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:34.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.983+0000 7f8a8e7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:34.985 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.983+0000 7f8a8e7fc700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a8008e8b0 0x7f8a80090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:34.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.983+0000 7f8a9e59c700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a8008e8b0 0x7f8a80090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:34.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.983+0000 7f8a8e7fc700 1 -- 172.21.15.143:0/2252168700 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f8a900b3d40 con 0x7f8aa006b680 2023-09-29T23:33:34.986 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.983+0000 7f8a9e59c700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a8008e8b0 0x7f8a80090d60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009d10 tx=0x7f8a940091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:34.991 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:34.987+0000 7f8a8e7fc700 1 -- 172.21.15.143:0/2252168700 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f8a90084790 con 0x7f8aa006b680 2023-09-29T23:33:35.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.183+0000 7f8a9f59e700 1 -- 172.21.15.143:0/2252168700 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8aa006cdb0 con 0x7f8aa006b680 2023-09-29T23:33:35.186 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.183+0000 7f8a8e7fc700 1 -- 172.21.15.143:0/2252168700 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f8a90017080 con 0x7f8aa006b680 2023-09-29T23:33:35.186 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:35.186 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:35.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 10 mgrc shutdown 2023-09-29T23:33:35.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 -- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a8008e8b0 msgr2=0x7f8a80090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:35.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a8008e8b0 0x7f8a80090d60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009d10 tx=0x7f8a940091a0 comp rx=0 tx=0).stop 2023-09-29T23:33:35.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 -- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa006b680 msgr2=0x7f8aa0132790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:35.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa006b680 0x7f8aa0132790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000c7e0 tx=0x7f8a90007e10 comp rx=0 tx=0).stop 2023-09-29T23:33:35.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 -- 172.21.15.143:0/2252168700 shutdown_connections 2023-09-29T23:33:35.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f8a8008e8b0 0x7f8a80090d60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:35.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f8aa0136520 0x7f8aa00b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:35.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8aa006b680 0x7f8aa0132790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:35.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 --2- 172.21.15.143:0/2252168700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8aa0066ec0 0x7f8aa0141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:35.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 -- 172.21.15.143:0/2252168700 >> 172.21.15.143:0/2252168700 conn(0x7f8aa006f200 msgr2=0x7f8aa00a3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:35.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 -- 172.21.15.143:0/2252168700 shutdown_connections 2023-09-29T23:33:35.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:35.187+0000 7f8a9f59e700 1 -- 172.21.15.143:0/2252168700 wait complete. 2023-09-29T23:33:38.205 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:38.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd223fff700 1 Processor -- start 2023-09-29T23:33:38.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd223fff700 1 -- start start 2023-09-29T23:33:38.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd223fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2241363f0 0x7fd2241367c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:38.319 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd223fff700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2240a5ed0 con 0x7fd2240a4f80 2023-09-29T23:33:38.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd223fff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2240a6720 con 0x7fd2240a6350 2023-09-29T23:33:38.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd223fff700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2240a5c10 con 0x7fd2241363f0 2023-09-29T23:33:38.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd2237fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fd2240a4f80 0x7fd2240a5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55290/0 (socket says 172.21.15.143:55290) 2023-09-29T23:33:38.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd2237fe700 1 -- 172.21.15.143:0/2351988723 learned_addr learned my addr 172.21.15.143:0/2351988723 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:33:38.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd2227fc700 1 --2- 172.21.15.143:0/2351988723 >> v2:172.21.15.143:3300/0 conn(0x7fd2241363f0 0x7fd2241367c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:38.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd221ffb700 1 -- 172.21.15.143:0/2351988723 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212942432 0 0) 0x7fd2240a5ed0 con 0x7fd2240a4f80 2023-09-29T23:33:38.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd221ffb700 1 -- 172.21.15.143:0/2351988723 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd20c003570 con 0x7fd2240a4f80 2023-09-29T23:33:38.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd2227fc700 1 -- 172.21.15.143:0/2351988723 >> v1:172.21.15.156:6789/0 conn(0x7fd2240a6350 legacy=0x7fd22406b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:38.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd221ffb700 1 -- 172.21.15.143:0/2351988723 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1158936371 0 0) 0x7fd2240a6720 con 0x7fd2240a6350 2023-09-29T23:33:38.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd2227fc700 1 -- 172.21.15.143:0/2351988723 >> v1:172.21.15.143:6789/0 conn(0x7fd2240a4f80 legacy=0x7fd2240a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:38.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd2227fc700 1 -- 172.21.15.143:0/2351988723 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22406ba70 con 0x7fd2241363f0 2023-09-29T23:33:38.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd2227fc700 1 --2- 172.21.15.143:0/2351988723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2241363f0 0x7fd2241367c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd214009e60 tx=0x7fd214009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afcb3adbcff6ea3 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:38.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd221ffb700 1 -- 172.21.15.143:0/2351988723 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21400e070 con 0x7fd2241363f0 2023-09-29T23:33:38.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd221ffb700 1 -- 172.21.15.143:0/2351988723 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21400bb60 con 0x7fd2241363f0 2023-09-29T23:33:38.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.315+0000 7fd221ffb700 1 -- 172.21.15.143:0/2351988723 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214012430 con 0x7fd2241363f0 2023-09-29T23:33:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 -- 172.21.15.143:0/2351988723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2241363f0 msgr2=0x7fd2241367c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 --2- 172.21.15.143:0/2351988723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2241363f0 0x7fd2241367c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd214009e60 tx=0x7fd214009780 comp rx=0 tx=0).stop 2023-09-29T23:33:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 -- 172.21.15.143:0/2351988723 shutdown_connections 2023-09-29T23:33:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 --2- 172.21.15.143:0/2351988723 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2241363f0 0x7fd2241367c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 -- 172.21.15.143:0/2351988723 >> 172.21.15.143:0/2351988723 conn(0x7fd2240b6930 msgr2=0x7fd2240b6d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:38.322 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 -- 172.21.15.143:0/2351988723 shutdown_connections 2023-09-29T23:33:38.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 -- 172.21.15.143:0/2351988723 wait complete. 2023-09-29T23:33:38.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 Processor -- start 2023-09-29T23:33:38.323 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 -- start start 2023-09-29T23:33:38.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2240a4f80 0x7fd22406ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:38.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2240a6350 0x7fd224130b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:38.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2240b3640 0x7fd2240b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:38.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd224153120 con 0x7fd2240b3640 2023-09-29T23:33:38.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240b4020 con 0x7fd2240a4f80 2023-09-29T23:33:38.325 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd223fff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22412fc50 con 0x7fd2240a6350 2023-09-29T23:33:38.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd2237fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2240b3640 0x7fd2240b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:38.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd2237fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2240b3640 0x7fd2240b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54162/0 (socket says 172.21.15.143:54162) 2023-09-29T23:33:38.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd2237fe700 1 -- 172.21.15.143:0/3906441273 learned_addr learned my addr 172.21.15.143:0/3906441273 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:38.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd2227fc700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2240a6350 0x7fd224130b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:38.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.319+0000 7fd222ffd700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2240a4f80 0x7fd22406ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:38.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd2237fe700 1 -- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2240a6350 msgr2=0x7fd224130b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:38.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd2237fe700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2240a6350 0x7fd224130b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:38.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd2237fe700 1 -- 172.21.15.143:0/3906441273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2240a4f80 msgr2=0x7fd22406ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:38.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd2237fe700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2240a4f80 0x7fd22406ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:38.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd2237fe700 1 -- 172.21.15.143:0/3906441273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd214009a20 con 0x7fd2240b3640 2023-09-29T23:33:38.326 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd222ffd700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2240a4f80 0x7fd22406ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:33:38.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd2237fe700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2240b3640 0x7fd2240b3ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00c790 tx=0x7fd21c007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:38.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd20affd700 1 -- 172.21.15.143:0/3906441273 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c005c00 con 0x7fd2240b3640 2023-09-29T23:33:38.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd20affd700 1 -- 172.21.15.143:0/3906441273 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21c005d60 con 0x7fd2240b3640 2023-09-29T23:33:38.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd20affd700 1 -- 172.21.15.143:0/3906441273 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0072e0 con 0x7fd2240b3640 2023-09-29T23:33:38.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd223fff700 1 -- 172.21.15.143:0/3906441273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd224130110 con 0x7fd2240b3640 2023-09-29T23:33:38.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd223fff700 1 -- 172.21.15.143:0/3906441273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd22412fe30 con 0x7fd2240b3640 2023-09-29T23:33:38.327 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd223fff700 1 -- 172.21.15.143:0/3906441273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd224145e80 con 0x7fd2240b3640 2023-09-29T23:33:38.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd20affd700 1 -- 172.21.15.143:0/3906441273 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fd21c012400 con 0x7fd2240b3640 2023-09-29T23:33:38.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.323+0000 7fd20affd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:38.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.327+0000 7fd20affd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:38.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.327+0000 7fd20affd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:38.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.327+0000 7fd20affd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:38.328 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.327+0000 7fd20affd700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd21008e8b0 0x7fd210090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:38.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.327+0000 7fd222ffd700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd21008e8b0 0x7fd210090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:38.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.327+0000 7fd20affd700 1 -- 172.21.15.143:0/3906441273 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fd21c0b4350 con 0x7fd2240b3640 2023-09-29T23:33:38.329 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.327+0000 7fd222ffd700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd21008e8b0 0x7fd210090d60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd218007fd0 tx=0x7fd218009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:38.335 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.331+0000 7fd20affd700 1 -- 172.21.15.143:0/3906441273 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd21c084c80 con 0x7fd2240b3640 2023-09-29T23:33:38.532 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 -- 172.21.15.143:0/3906441273 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd22406a5f0 con 0x7fd2240b3640 2023-09-29T23:33:38.533 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd20affd700 1 -- 172.21.15.143:0/3906441273 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fd21c021100 con 0x7fd2240b3640 2023-09-29T23:33:38.533 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:38.533 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:38.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 10 mgrc shutdown 2023-09-29T23:33:38.536 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 -- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd21008e8b0 msgr2=0x7fd210090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:38.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd21008e8b0 0x7fd210090d60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd218007fd0 tx=0x7fd218009040 comp rx=0 tx=0).stop 2023-09-29T23:33:38.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 -- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2240b3640 msgr2=0x7fd2240b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:38.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2240b3640 0x7fd2240b3ae0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd21c00c790 tx=0x7fd21c007dd0 comp rx=0 tx=0).stop 2023-09-29T23:33:38.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 -- 172.21.15.143:0/3906441273 shutdown_connections 2023-09-29T23:33:38.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd21008e8b0 0x7fd210090d60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:38.537 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2240b3640 0x7fd2240b3ae0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:38.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2240a6350 0x7fd224130b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:38.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 --2- 172.21.15.143:0/3906441273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2240a4f80 0x7fd22406ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:38.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 -- 172.21.15.143:0/3906441273 >> 172.21.15.143:0/3906441273 conn(0x7fd2240b6930 msgr2=0x7fd224145770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:38.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 -- 172.21.15.143:0/3906441273 shutdown_connections 2023-09-29T23:33:38.538 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:38.531+0000 7fd223fff700 1 -- 172.21.15.143:0/3906441273 wait complete. 2023-09-29T23:33:41.548 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:41.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.659+0000 7f7c6514e700 1 Processor -- start 2023-09-29T23:33:41.663 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.659+0000 7f7c6514e700 1 -- start start 2023-09-29T23:33:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.659+0000 7f7c6514e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7c600684c0 0x7f7c60066e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.659+0000 7f7c6514e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c6009f450 0x7f7c6006bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.659+0000 7f7c6514e700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6006c040 con 0x7f7c60132210 2023-09-29T23:33:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.659+0000 7f7c6514e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c600a0e50 con 0x7f7c600684c0 2023-09-29T23:33:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.659+0000 7f7c6514e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600a10d0 con 0x7f7c6009f450 2023-09-29T23:33:41.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c5f7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c6009f450 0x7f7c6006bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6494d700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f7c60132210 0x7f7c6009fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55300/0 (socket says 172.21.15.143:55300) 2023-09-29T23:33:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6494d700 1 -- 172.21.15.143:0/1395966140 learned_addr learned my addr 172.21.15.143:0/1395966140 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:33:41.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c5ffff700 1 --2- 172.21.15.143:0/1395966140 >> v2:172.21.15.156:3300/0 conn(0x7f7c600684c0 0x7f7c60066e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:41.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c5f7fe700 1 -- 172.21.15.143:0/1395966140 >> v2:172.21.15.156:3300/0 conn(0x7f7c600684c0 msgr2=0x7f7c60066e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:41.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c5f7fe700 1 --2- 172.21.15.143:0/1395966140 >> v2:172.21.15.156:3300/0 conn(0x7f7c600684c0 0x7f7c60066e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:41.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c5f7fe700 1 -- 172.21.15.143:0/1395966140 >> v1:172.21.15.143:6789/0 conn(0x7f7c60132210 legacy=0x7f7c6009fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:41.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c5f7fe700 1 -- 172.21.15.143:0/1395966140 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c600a1390 con 0x7f7c6009f450 2023-09-29T23:33:41.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c5f7fe700 1 --2- 172.21.15.143:0/1395966140 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c6009f450 0x7f7c6006bb00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7c540098e0 tx=0x7f7c540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4b2eff9cc8912cf server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:41.666 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c5effd700 1 -- 172.21.15.143:0/1395966140 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5400b640 con 0x7f7c6009f450 2023-09-29T23:33:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c5effd700 1 -- 172.21.15.143:0/1395966140 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5400b7a0 con 0x7f7c6009f450 2023-09-29T23:33:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c5effd700 1 -- 172.21.15.143:0/1395966140 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5400bb40 con 0x7f7c6009f450 2023-09-29T23:33:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6514e700 1 -- 172.21.15.143:0/1395966140 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c6009f450 msgr2=0x7f7c6006bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6514e700 1 --2- 172.21.15.143:0/1395966140 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c6009f450 0x7f7c6006bb00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7c540098e0 tx=0x7f7c540091a0 comp rx=0 tx=0).stop 2023-09-29T23:33:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6514e700 1 -- 172.21.15.143:0/1395966140 shutdown_connections 2023-09-29T23:33:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6514e700 1 --2- 172.21.15.143:0/1395966140 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c6009f450 0x7f7c6006bb00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6514e700 1 --2- 172.21.15.143:0/1395966140 >> v2:172.21.15.156:3300/0 conn(0x7f7c600684c0 0x7f7c60066e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6514e700 1 -- 172.21.15.143:0/1395966140 >> 172.21.15.143:0/1395966140 conn(0x7f7c6006f200 msgr2=0x7f7c60069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6514e700 1 -- 172.21.15.143:0/1395966140 shutdown_connections 2023-09-29T23:33:41.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6514e700 1 -- 172.21.15.143:0/1395966140 wait complete. 2023-09-29T23:33:41.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6514e700 1 Processor -- start 2023-09-29T23:33:41.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.663+0000 7f7c6514e700 1 -- start start 2023-09-29T23:33:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c6514e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c600684c0 0x7f7c60099ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c6514e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c6009f450 0x7f7c600af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c6514e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c60132210 0x7f7c600afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c6514e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6014d860 con 0x7f7c60132210 2023-09-29T23:33:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c6514e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6014db20 con 0x7f7c6009f450 2023-09-29T23:33:41.669 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c6514e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6009baf0 con 0x7f7c600684c0 2023-09-29T23:33:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c5ffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c600684c0 0x7f7c60099ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c6494d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c60132210 0x7f7c600afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c5ffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c600684c0 0x7f7c60099ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:39200/0 (socket says 172.21.15.143:39200) 2023-09-29T23:33:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c5ffff700 1 -- 172.21.15.143:0/1452554331 learned_addr learned my addr 172.21.15.143:0/1452554331 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c5f7fe700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c6009f450 0x7f7c600af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c5ffff700 1 -- 172.21.15.143:0/1452554331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c6009f450 msgr2=0x7f7c600af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c5ffff700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c6009f450 0x7f7c600af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c5ffff700 1 -- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c60132210 msgr2=0x7f7c600afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c5ffff700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c60132210 0x7f7c600afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c5ffff700 1 -- 172.21.15.143:0/1452554331 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c54009420 con 0x7f7c600684c0 2023-09-29T23:33:41.670 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c6494d700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c60132210 0x7f7c600afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:33:41.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c5ffff700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c600684c0 0x7f7c60099ce0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009ef0 tx=0x7f7c5800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:41.671 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c47fff700 1 -- 172.21.15.143:0/1452554331 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5800cb70 con 0x7f7c600684c0 2023-09-29T23:33:41.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c47fff700 1 -- 172.21.15.143:0/1452554331 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5800f040 con 0x7f7c600684c0 2023-09-29T23:33:41.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c6514e700 1 -- 172.21.15.143:0/1452554331 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c60097c30 con 0x7f7c600684c0 2023-09-29T23:33:41.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c47fff700 1 -- 172.21.15.143:0/1452554331 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c58014480 con 0x7f7c600684c0 2023-09-29T23:33:41.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c6514e700 1 -- 172.21.15.143:0/1452554331 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c60098a20 con 0x7f7c600684c0 2023-09-29T23:33:41.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.667+0000 7f7c6514e700 1 -- 172.21.15.143:0/1452554331 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6005ecb0 con 0x7f7c600684c0 2023-09-29T23:33:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.671+0000 7f7c47fff700 1 -- 172.21.15.143:0/1452554331 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f7c580147f0 con 0x7f7c600684c0 2023-09-29T23:33:41.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.671+0000 7f7c47fff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.671+0000 7f7c47fff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.671+0000 7f7c47fff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:41.674 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.671+0000 7f7c47fff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.671+0000 7f7c47fff700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c4c08e8b0 0x7f7c4c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.671+0000 7f7c47fff700 1 -- 172.21.15.143:0/1452554331 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f7c580b5890 con 0x7f7c600684c0 2023-09-29T23:33:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.671+0000 7f7c5f7fe700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c4c08e8b0 0x7f7c4c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:41.675 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.671+0000 7f7c5f7fe700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c4c08e8b0 0x7f7c4c090d60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7c540093f0 tx=0x7f7c54017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:41.683 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.679+0000 7f7c47fff700 1 -- 172.21.15.143:0/1452554331 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f7c580862e0 con 0x7f7c600684c0 2023-09-29T23:33:41.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.879+0000 7f7c6514e700 1 -- 172.21.15.143:0/1452554331 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7c600a0330 con 0x7f7c600684c0 2023-09-29T23:33:41.884 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c47fff700 1 -- 172.21.15.143:0/1452554331 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f7c58089900 con 0x7f7c600684c0 2023-09-29T23:33:41.884 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:41.885 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 10 mgrc shutdown 2023-09-29T23:33:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 -- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c4c08e8b0 msgr2=0x7f7c4c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c4c08e8b0 0x7f7c4c090d60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7c540093f0 tx=0x7f7c54017000 comp rx=0 tx=0).stop 2023-09-29T23:33:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 -- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c600684c0 msgr2=0x7f7c60099ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c600684c0 0x7f7c60099ce0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009ef0 tx=0x7f7c5800b040 comp rx=0 tx=0).stop 2023-09-29T23:33:41.887 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 -- 172.21.15.143:0/1452554331 shutdown_connections 2023-09-29T23:33:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f7c4c08e8b0 0x7f7c4c090d60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c60132210 0x7f7c600afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c6009f450 0x7f7c600af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 --2- 172.21.15.143:0/1452554331 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7c600684c0 0x7f7c60099ce0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 -- 172.21.15.143:0/1452554331 >> 172.21.15.143:0/1452554331 conn(0x7f7c6006f200 msgr2=0x7f7c6006b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 -- 172.21.15.143:0/1452554331 shutdown_connections 2023-09-29T23:33:41.888 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:41.883+0000 7f7c6514e700 1 -- 172.21.15.143:0/1452554331 wait complete. 2023-09-29T23:33:44.899 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:45.010 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.007+0000 7f98e73c5700 1 Processor -- start 2023-09-29T23:33:45.011 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.007+0000 7f98e73c5700 1 -- start start 2023-09-29T23:33:45.012 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.007+0000 7f98e73c5700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a5f30 0x7f98e00a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.007+0000 7f98e73c5700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f98e013d630 0x7f98e013da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.007+0000 7f98e73c5700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e00a2760 con 0x7f98e00a3150 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.007+0000 7f98e73c5700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e013ce60 con 0x7f98e013d630 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.007+0000 7f98e73c5700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e00a1950 con 0x7f98e00a5f30 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.007+0000 7f98e5bc2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a5f30 0x7f98e00a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.007+0000 7f98e5bc2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a5f30 0x7f98e00a0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:40962/0 (socket says 172.21.15.143:40962) 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e5bc2700 1 -- 172.21.15.143:0/2623654668 learned_addr learned my addr 172.21.15.143:0/2623654668 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.007+0000 7f98e63c3700 1 --1- 172.21.15.143:0/2623654668 >> v1:172.21.15.143:6789/0 conn(0x7f98e00a3150 0x7f98e00a2050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56420/0 (socket says 172.21.15.143:56420) 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e6bc4700 1 --2- 172.21.15.143:0/2623654668 >> v2:172.21.15.156:3300/0 conn(0x7f98e013d630 0x7f98e013da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e5bc2700 1 -- 172.21.15.143:0/2623654668 >> v2:172.21.15.156:3300/0 conn(0x7f98e013d630 msgr2=0x7f98e013da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e5bc2700 1 --2- 172.21.15.143:0/2623654668 >> v2:172.21.15.156:3300/0 conn(0x7f98e013d630 0x7f98e013da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:45.013 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e5bc2700 1 -- 172.21.15.143:0/2623654668 >> v1:172.21.15.143:6789/0 conn(0x7f98e00a3150 legacy=0x7f98e00a2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:45.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e5bc2700 1 -- 172.21.15.143:0/2623654668 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e00a29f0 con 0x7f98e00a5f30 2023-09-29T23:33:45.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e6bc4700 1 --2- 172.21.15.143:0/2623654668 >> v2:172.21.15.156:3300/0 conn(0x7f98e013d630 0x7f98e013da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:33:45.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e5bc2700 1 --2- 172.21.15.143:0/2623654668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a5f30 0x7f98e00a0ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f98d40099b0 tx=0x7f98d4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eae1b5e8e558936b server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:45.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e53c1700 1 -- 172.21.15.143:0/2623654668 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d400b860 con 0x7f98e00a5f30 2023-09-29T23:33:45.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e53c1700 1 -- 172.21.15.143:0/2623654668 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d400b9c0 con 0x7f98e00a5f30 2023-09-29T23:33:45.014 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e53c1700 1 -- 172.21.15.143:0/2623654668 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d400bd60 con 0x7f98e00a5f30 2023-09-29T23:33:45.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e73c5700 1 -- 172.21.15.143:0/2623654668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a5f30 msgr2=0x7f98e00a0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:45.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e73c5700 1 --2- 172.21.15.143:0/2623654668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a5f30 0x7f98e00a0ef0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f98d40099b0 tx=0x7f98d4009250 comp rx=0 tx=0).stop 2023-09-29T23:33:45.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e73c5700 1 -- 172.21.15.143:0/2623654668 shutdown_connections 2023-09-29T23:33:45.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e73c5700 1 --2- 172.21.15.143:0/2623654668 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a5f30 0x7f98e00a0ef0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:45.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e73c5700 1 --2- 172.21.15.143:0/2623654668 >> v2:172.21.15.156:3300/0 conn(0x7f98e013d630 0x7f98e013da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:45.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e73c5700 1 -- 172.21.15.143:0/2623654668 >> 172.21.15.143:0/2623654668 conn(0x7f98e0067da0 msgr2=0x7f98e00681a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:45.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e73c5700 1 -- 172.21.15.143:0/2623654668 shutdown_connections 2023-09-29T23:33:45.015 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.011+0000 7f98e73c5700 1 -- 172.21.15.143:0/2623654668 wait complete. 2023-09-29T23:33:45.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e73c5700 1 Processor -- start 2023-09-29T23:33:45.016 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e73c5700 1 -- start start 2023-09-29T23:33:45.018 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e73c5700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a3150 0x7f98e0066ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:45.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e73c5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e00a5f30 0x7f98e0151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:45.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e73c5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e013d630 0x7f98e009bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:45.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e73c5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0151b90 con 0x7f98e013d630 2023-09-29T23:33:45.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e73c5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e0151e50 con 0x7f98e00a5f30 2023-09-29T23:33:45.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e73c5700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e014f030 con 0x7f98e00a3150 2023-09-29T23:33:45.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e6bc4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e013d630 0x7f98e009bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:45.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e63c3700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a3150 0x7f98e0066ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:45.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e6bc4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e013d630 0x7f98e009bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49602/0 (socket says 172.21.15.143:49602) 2023-09-29T23:33:45.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e6bc4700 1 -- 172.21.15.143:0/4155725943 learned_addr learned my addr 172.21.15.143:0/4155725943 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:45.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e5bc2700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e00a5f30 0x7f98e0151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:45.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e6bc4700 1 -- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a3150 msgr2=0x7f98e0066ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:45.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e6bc4700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a3150 0x7f98e0066ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:45.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e6bc4700 1 -- 172.21.15.143:0/4155725943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e00a5f30 msgr2=0x7f98e0151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:45.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e6bc4700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e00a5f30 0x7f98e0151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:45.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e6bc4700 1 -- 172.21.15.143:0/4155725943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d40094f0 con 0x7f98e013d630 2023-09-29T23:33:45.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e5bc2700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e00a5f30 0x7f98e0151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:33:45.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e6bc4700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e013d630 0x7f98e009bf50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f98d000e870 tx=0x7f98d0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:45.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98de7fc700 1 -- 172.21.15.143:0/4155725943 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0013410 con 0x7f98e013d630 2023-09-29T23:33:45.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98de7fc700 1 -- 172.21.15.143:0/4155725943 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d0013570 con 0x7f98e013d630 2023-09-29T23:33:45.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98de7fc700 1 -- 172.21.15.143:0/4155725943 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0013910 con 0x7f98e013d630 2023-09-29T23:33:45.020 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e73c5700 1 -- 172.21.15.143:0/4155725943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e009b9b0 con 0x7f98e013d630 2023-09-29T23:33:45.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.015+0000 7f98e73c5700 1 -- 172.21.15.143:0/4155725943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e00674f0 con 0x7f98e013d630 2023-09-29T23:33:45.021 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.019+0000 7f98e73c5700 1 -- 172.21.15.143:0/4155725943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e005ecb0 con 0x7f98e013d630 2023-09-29T23:33:45.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.019+0000 7f98de7fc700 1 -- 172.21.15.143:0/4155725943 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f98d0013cc0 con 0x7f98e013d630 2023-09-29T23:33:45.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.019+0000 7f98de7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:45.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.019+0000 7f98de7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:45.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.019+0000 7f98de7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:45.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.019+0000 7f98de7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:45.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.019+0000 7f98de7fc700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f98cc08e8b0 0x7f98cc090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:45.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.019+0000 7f98de7fc700 1 -- 172.21.15.143:0/4155725943 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f98d00b43a0 con 0x7f98e013d630 2023-09-29T23:33:45.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.019+0000 7f98e63c3700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f98cc08e8b0 0x7f98cc090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:45.024 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.023+0000 7f98e63c3700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f98cc08e8b0 0x7f98cc090d60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98d8002020 tx=0x7f98d8001a30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:45.029 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.027+0000 7f98de7fc700 1 -- 172.21.15.143:0/4155725943 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f98d0084df0 con 0x7f98e013d630 2023-09-29T23:33:45.223 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.219+0000 7f98e73c5700 1 -- 172.21.15.143:0/4155725943 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f98e013d050 con 0x7f98e013d630 2023-09-29T23:33:45.224 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98de7fc700 1 -- 172.21.15.143:0/4155725943 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f98d0088410 con 0x7f98e013d630 2023-09-29T23:33:45.224 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:45.224 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 10 mgrc shutdown 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 -- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f98cc08e8b0 msgr2=0x7f98cc090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f98cc08e8b0 0x7f98cc090d60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f98d8002020 tx=0x7f98d8001a30 comp rx=0 tx=0).stop 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 -- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e013d630 msgr2=0x7f98e009bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e013d630 0x7f98e009bf50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f98d000e870 tx=0x7f98d0008040 comp rx=0 tx=0).stop 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 -- 172.21.15.143:0/4155725943 shutdown_connections 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f98cc08e8b0 0x7f98cc090d60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98e013d630 0x7f98e009bf50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e00a5f30 0x7f98e0151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 --2- 172.21.15.143:0/4155725943 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f98e00a3150 0x7f98e0066ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 -- 172.21.15.143:0/4155725943 >> 172.21.15.143:0/4155725943 conn(0x7f98e0067da0 msgr2=0x7f98e00b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:45.227 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 -- 172.21.15.143:0/4155725943 shutdown_connections 2023-09-29T23:33:45.228 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:45.223+0000 7f98e73c5700 1 -- 172.21.15.143:0/4155725943 wait complete. 2023-09-29T23:33:48.239 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:48.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.338+0000 7f4c81864700 1 Processor -- start 2023-09-29T23:33:48.343 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.338+0000 7f4c81864700 1 -- start start 2023-09-29T23:33:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c81864700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 0x7f4c7c09b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c81864700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4c7c13f6a0 0x7f4c7c13fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c81864700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7c06f430 con 0x7f4c7c143b00 2023-09-29T23:33:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c81864700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7c09a490 con 0x7f4c7c13f6a0 2023-09-29T23:33:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c81864700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c09b630 con 0x7f4c7c06f060 2023-09-29T23:33:48.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c7a7fc700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f4c7c143b00 0x7f4c7c143ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56428/0 (socket says 172.21.15.143:56428) 2023-09-29T23:33:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c7a7fc700 1 -- 172.21.15.143:0/2852576149 learned_addr learned my addr 172.21.15.143:0/2852576149 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:33:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c7affd700 1 --2- 172.21.15.143:0/2852576149 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 0x7f4c7c09b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:48.346 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c7b7fe700 1 --2- 172.21.15.143:0/2852576149 >> v2:172.21.15.156:3300/0 conn(0x7f4c7c13f6a0 0x7f4c7c13fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c79ffb700 1 -- 172.21.15.143:0/2852576149 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563332879 0 0) 0x7f4c7c06f430 con 0x7f4c7c143b00 2023-09-29T23:33:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c79ffb700 1 -- 172.21.15.143:0/2852576149 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c64003570 con 0x7f4c7c143b00 2023-09-29T23:33:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c7affd700 1 -- 172.21.15.143:0/2852576149 >> v2:172.21.15.156:3300/0 conn(0x7f4c7c13f6a0 msgr2=0x7f4c7c13fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c7affd700 1 --2- 172.21.15.143:0/2852576149 >> v2:172.21.15.156:3300/0 conn(0x7f4c7c13f6a0 0x7f4c7c13fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c7affd700 1 -- 172.21.15.143:0/2852576149 >> v1:172.21.15.143:6789/0 conn(0x7f4c7c143b00 legacy=0x7f4c7c143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c7affd700 1 -- 172.21.15.143:0/2852576149 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c099e50 con 0x7f4c7c06f060 2023-09-29T23:33:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c7affd700 1 --2- 172.21.15.143:0/2852576149 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 0x7f4c7c09b0f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0098e0 tx=0x7f4c6c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b45d996d8894285a server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c79ffb700 1 -- 172.21.15.143:0/2852576149 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c011630 con 0x7f4c7c06f060 2023-09-29T23:33:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c79ffb700 1 -- 172.21.15.143:0/2852576149 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c011790 con 0x7f4c7c06f060 2023-09-29T23:33:48.347 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c79ffb700 1 -- 172.21.15.143:0/2852576149 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c011b30 con 0x7f4c7c06f060 2023-09-29T23:33:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c81864700 1 -- 172.21.15.143:0/2852576149 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 msgr2=0x7f4c7c09b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.342+0000 7f4c81864700 1 --2- 172.21.15.143:0/2852576149 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 0x7f4c7c09b0f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0098e0 tx=0x7f4c6c0091a0 comp rx=0 tx=0).stop 2023-09-29T23:33:48.348 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 -- 172.21.15.143:0/2852576149 shutdown_connections 2023-09-29T23:33:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 --2- 172.21.15.143:0/2852576149 >> v2:172.21.15.156:3300/0 conn(0x7f4c7c13f6a0 0x7f4c7c13fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 --2- 172.21.15.143:0/2852576149 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 0x7f4c7c09b0f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 -- 172.21.15.143:0/2852576149 >> 172.21.15.143:0/2852576149 conn(0x7f4c7c0b2ee0 msgr2=0x7f4c7c0b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 -- 172.21.15.143:0/2852576149 shutdown_connections 2023-09-29T23:33:48.349 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 -- 172.21.15.143:0/2852576149 wait complete. 2023-09-29T23:33:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 Processor -- start 2023-09-29T23:33:48.350 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 -- start start 2023-09-29T23:33:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 0x7f4c7c06d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c7c13f6a0 0x7f4c7c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c7c143b00 0x7f4c7c06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c06cdb0 con 0x7f4c7c13f6a0 2023-09-29T23:33:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c06cb40 con 0x7f4c7c143b00 2023-09-29T23:33:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.346+0000 7f4c81864700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c06d9b0 con 0x7f4c7c06f060 2023-09-29T23:33:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7affd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 0x7f4c7c06d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7a7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c7c13f6a0 0x7f4c7c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7affd700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 0x7f4c7c06d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:40996/0 (socket says 172.21.15.143:40996) 2023-09-29T23:33:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7affd700 1 -- 172.21.15.143:0/1325813841 learned_addr learned my addr 172.21.15.143:0/1325813841 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:48.353 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7b7fe700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c7c143b00 0x7f4c7c06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7affd700 1 -- 172.21.15.143:0/1325813841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c7c143b00 msgr2=0x7f4c7c06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7affd700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c7c143b00 0x7f4c7c06b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7affd700 1 -- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c7c13f6a0 msgr2=0x7f4c7c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7affd700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c7c13f6a0 0x7f4c7c06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7affd700 1 -- 172.21.15.143:0/1325813841 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c009420 con 0x7f4c7c06f060 2023-09-29T23:33:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7b7fe700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c7c143b00 0x7f4c7c06b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:33:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c7affd700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 0x7f4c7c06d2d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c068e90 tx=0x7f4c6c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c62ffd700 1 -- 172.21.15.143:0/1325813841 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c011b10 con 0x7f4c7c06f060 2023-09-29T23:33:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c62ffd700 1 -- 172.21.15.143:0/1325813841 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c000cc0 con 0x7f4c7c06f060 2023-09-29T23:33:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c81864700 1 -- 172.21.15.143:0/1325813841 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7c06dc70 con 0x7f4c7c06f060 2023-09-29T23:33:48.354 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c62ffd700 1 -- 172.21.15.143:0/1325813841 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c023600 con 0x7f4c7c06f060 2023-09-29T23:33:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c81864700 1 -- 172.21.15.143:0/1325813841 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c06bcf0 con 0x7f4c7c06f060 2023-09-29T23:33:48.355 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.350+0000 7f4c81864700 1 -- 172.21.15.143:0/1325813841 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c05ecb0 con 0x7f4c7c06f060 2023-09-29T23:33:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.354+0000 7f4c62ffd700 1 -- 172.21.15.143:0/1325813841 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f4c6c02c3b0 con 0x7f4c7c06f060 2023-09-29T23:33:48.357 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.354+0000 7f4c62ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.354+0000 7f4c62ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.354+0000 7f4c62ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.354+0000 7f4c62ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.354+0000 7f4c62ffd700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c6808e8b0 0x7f4c68090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.354+0000 7f4c7a7fc700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c6808e8b0 0x7f4c68090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:48.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.354+0000 7f4c62ffd700 1 -- 172.21.15.143:0/1325813841 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f4c6c0b5c90 con 0x7f4c7c06f060 2023-09-29T23:33:48.359 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.354+0000 7f4c7a7fc700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c6808e8b0 0x7f4c68090d60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c06bfe0 tx=0x7f4c70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:48.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.362+0000 7f4c62ffd700 1 -- 172.21.15.143:0/1325813841 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4c6c086640 con 0x7f4c7c06f060 2023-09-29T23:33:48.568 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.566+0000 7f4c81864700 1 -- 172.21.15.143:0/1325813841 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f4c7c140040 con 0x7f4c7c06f060 2023-09-29T23:33:48.569 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.566+0000 7f4c62ffd700 1 -- 172.21.15.143:0/1325813841 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f4c6c089c60 con 0x7f4c7c06f060 2023-09-29T23:33:48.569 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:48.570 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:48.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 10 mgrc shutdown 2023-09-29T23:33:48.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 -- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c6808e8b0 msgr2=0x7f4c68090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:48.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c6808e8b0 0x7f4c68090d60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c06bfe0 tx=0x7f4c70009040 comp rx=0 tx=0).stop 2023-09-29T23:33:48.574 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 -- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 msgr2=0x7f4c7c06d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:48.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 0x7f4c7c06d2d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c068e90 tx=0x7f4c6c017000 comp rx=0 tx=0).stop 2023-09-29T23:33:48.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 -- 172.21.15.143:0/1325813841 shutdown_connections 2023-09-29T23:33:48.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f4c6808e8b0 0x7f4c68090d60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:48.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c7c143b00 0x7f4c7c06b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:48.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4c7c13f6a0 0x7f4c7c06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:48.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 --2- 172.21.15.143:0/1325813841 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4c7c06f060 0x7f4c7c06d2d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:48.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 -- 172.21.15.143:0/1325813841 >> 172.21.15.143:0/1325813841 conn(0x7f4c7c0b2ee0 msgr2=0x7f4c7c0b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:48.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 -- 172.21.15.143:0/1325813841 shutdown_connections 2023-09-29T23:33:48.575 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:48.570+0000 7f4c81864700 1 -- 172.21.15.143:0/1325813841 wait complete. 2023-09-29T23:33:51.590 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:51.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.698+0000 7f2fb29a3700 1 Processor -- start 2023-09-29T23:33:51.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.698+0000 7f2fb29a3700 1 -- start start 2023-09-29T23:33:51.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.698+0000 7f2fb29a3700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 0x7f2fac13d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:51.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.698+0000 7f2fb29a3700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac09ff40 con 0x7f2fac13d680 2023-09-29T23:33:51.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.698+0000 7f2fb29a3700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fac067290 con 0x7f2fac066ec0 2023-09-29T23:33:51.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.698+0000 7f2fb29a3700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac0a0510 con 0x7f2fac0a1170 2023-09-29T23:33:51.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.698+0000 7f2fb11a0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 0x7f2fac13d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:51.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.698+0000 7f2fb21a2700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f2fac13d680 0x7f2fac148590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:56438/0 (socket says 172.21.15.143:56438) 2023-09-29T23:33:51.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.698+0000 7f2fb21a2700 1 -- 172.21.15.143:0/210411271 learned_addr learned my addr 172.21.15.143:0/210411271 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:33:51.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb099f700 1 -- 172.21.15.143:0/210411271 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555723301 0 0) 0x7f2fac09ff40 con 0x7f2fac13d680 2023-09-29T23:33:51.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb099f700 1 -- 172.21.15.143:0/210411271 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f98003570 con 0x7f2fac13d680 2023-09-29T23:33:51.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb099f700 1 -- 172.21.15.143:0/210411271 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378931592 0 0) 0x7f2fac067290 con 0x7f2fac066ec0 2023-09-29T23:33:51.705 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb11a0700 1 -- 172.21.15.143:0/210411271 >> v1:172.21.15.156:6789/0 conn(0x7f2fac066ec0 legacy=0x7f2fac09f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:51.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb11a0700 1 -- 172.21.15.143:0/210411271 >> v1:172.21.15.143:6789/0 conn(0x7f2fac13d680 legacy=0x7f2fac148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:51.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb11a0700 1 -- 172.21.15.143:0/210411271 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fac06e400 con 0x7f2fac0a1170 2023-09-29T23:33:51.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb11a0700 1 --2- 172.21.15.143:0/210411271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 0x7f2fac13d140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009e90 tx=0x7f2fa8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c92bd631154a5c01 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:51.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb099f700 1 -- 172.21.15.143:0/210411271 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800e070 con 0x7f2fac0a1170 2023-09-29T23:33:51.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb099f700 1 -- 172.21.15.143:0/210411271 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa800bc30 con 0x7f2fac0a1170 2023-09-29T23:33:51.706 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb099f700 1 -- 172.21.15.143:0/210411271 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8012430 con 0x7f2fac0a1170 2023-09-29T23:33:51.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb29a3700 1 -- 172.21.15.143:0/210411271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 msgr2=0x7f2fac13d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:51.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb29a3700 1 --2- 172.21.15.143:0/210411271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 0x7f2fac13d140 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009e90 tx=0x7f2fa8009780 comp rx=0 tx=0).stop 2023-09-29T23:33:51.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb29a3700 1 -- 172.21.15.143:0/210411271 shutdown_connections 2023-09-29T23:33:51.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb29a3700 1 --2- 172.21.15.143:0/210411271 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 0x7f2fac13d140 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:51.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb29a3700 1 -- 172.21.15.143:0/210411271 >> 172.21.15.143:0/210411271 conn(0x7f2fac06f200 msgr2=0x7f2fac069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:51.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb29a3700 1 -- 172.21.15.143:0/210411271 shutdown_connections 2023-09-29T23:33:51.707 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.702+0000 7f2fb29a3700 1 -- 172.21.15.143:0/210411271 wait complete. 2023-09-29T23:33:51.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb29a3700 1 Processor -- start 2023-09-29T23:33:51.709 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb29a3700 1 -- start start 2023-09-29T23:33:51.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb29a3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fac066ec0 0x7f2fac0988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:51.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb29a3700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 0x7f2fac13fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:51.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb29a3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fac13d680 0x7f2fac140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:51.711 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb29a3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac098200 con 0x7f2fac066ec0 2023-09-29T23:33:51.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb29a3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac1408b0 con 0x7f2fac13d680 2023-09-29T23:33:51.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb29a3700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fac140b70 con 0x7f2fac0a1170 2023-09-29T23:33:51.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb11a0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 0x7f2fac13fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:51.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb19a1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fac066ec0 0x7f2fac0988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:51.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb11a0700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 0x7f2fac13fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:41020/0 (socket says 172.21.15.143:41020) 2023-09-29T23:33:51.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb11a0700 1 -- 172.21.15.143:0/3664059814 learned_addr learned my addr 172.21.15.143:0/3664059814 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:51.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb21a2700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fac13d680 0x7f2fac140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:51.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb19a1700 1 -- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 msgr2=0x7f2fac13fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:51.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb19a1700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 0x7f2fac13fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:51.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb19a1700 1 -- 172.21.15.143:0/3664059814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fac13d680 msgr2=0x7f2fac140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:51.712 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb19a1700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fac13d680 0x7f2fac140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:51.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb19a1700 1 -- 172.21.15.143:0/3664059814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa8009a20 con 0x7f2fac066ec0 2023-09-29T23:33:51.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb11a0700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 0x7f2fac13fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:33:51.713 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.706+0000 7f2fb19a1700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fac066ec0 0x7f2fac0988c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0011e0 tx=0x7f2f9c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:51.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.710+0000 7f2fa1ffb700 1 -- 172.21.15.143:0/3664059814 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c011590 con 0x7f2fac066ec0 2023-09-29T23:33:51.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.710+0000 7f2fa1ffb700 1 -- 172.21.15.143:0/3664059814 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f9c011710 con 0x7f2fac066ec0 2023-09-29T23:33:51.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.710+0000 7f2fb29a3700 1 -- 172.21.15.143:0/3664059814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fac1419a0 con 0x7f2fac066ec0 2023-09-29T23:33:51.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.710+0000 7f2fa1ffb700 1 -- 172.21.15.143:0/3664059814 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c011ad0 con 0x7f2fac066ec0 2023-09-29T23:33:51.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.710+0000 7f2fb29a3700 1 -- 172.21.15.143:0/3664059814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fac098fa0 con 0x7f2fac066ec0 2023-09-29T23:33:51.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.710+0000 7f2fb29a3700 1 -- 172.21.15.143:0/3664059814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac05ecb0 con 0x7f2fac066ec0 2023-09-29T23:33:51.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.710+0000 7f2fa1ffb700 1 -- 172.21.15.143:0/3664059814 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f2f9c01b3b0 con 0x7f2fac066ec0 2023-09-29T23:33:51.715 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.710+0000 7f2fa1ffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:51.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.714+0000 7f2fa1ffb700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:51.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.714+0000 7f2fa1ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:51.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.714+0000 7f2fa1ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:51.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.714+0000 7f2fa1ffb700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f9008e8b0 0x7f2f90090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:51.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.714+0000 7f2fa1ffb700 1 -- 172.21.15.143:0/3664059814 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f2f9c0b40f0 con 0x7f2fac066ec0 2023-09-29T23:33:51.716 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.714+0000 7f2fb11a0700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f9008e8b0 0x7f2f90090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:51.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.714+0000 7f2fb11a0700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f9008e8b0 0x7f2f90090d60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2fac141510 tx=0x7f2fa8019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:51.723 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.718+0000 7f2fa1ffb700 1 -- 172.21.15.143:0/3664059814 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f2f9c084ab0 con 0x7f2fac066ec0 2023-09-29T23:33:51.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.922+0000 7f2fb29a3700 1 -- 172.21.15.143:0/3664059814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f2fac06e6c0 con 0x7f2fac066ec0 2023-09-29T23:33:51.925 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.922+0000 7f2fa1ffb700 1 -- 172.21.15.143:0/3664059814 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f2f9c017020 con 0x7f2fac066ec0 2023-09-29T23:33:51.926 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:51.926 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:51.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.922+0000 7f2fb29a3700 10 mgrc shutdown 2023-09-29T23:33:51.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 -- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f9008e8b0 msgr2=0x7f2f90090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:51.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f9008e8b0 0x7f2f90090d60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2fac141510 tx=0x7f2fa8019000 comp rx=0 tx=0).stop 2023-09-29T23:33:51.928 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 -- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fac066ec0 msgr2=0x7f2fac0988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fac066ec0 0x7f2fac0988c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c0011e0 tx=0x7f2f9c007d30 comp rx=0 tx=0).stop 2023-09-29T23:33:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 -- 172.21.15.143:0/3664059814 shutdown_connections 2023-09-29T23:33:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f2f9008e8b0 0x7f2f90090d60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fac13d680 0x7f2fac140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f2fac0a1170 0x7f2fac13fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 --2- 172.21.15.143:0/3664059814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fac066ec0 0x7f2fac0988c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 -- 172.21.15.143:0/3664059814 >> 172.21.15.143:0/3664059814 conn(0x7f2fac06f200 msgr2=0x7f2fac13b340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 -- 172.21.15.143:0/3664059814 shutdown_connections 2023-09-29T23:33:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:51.926+0000 7f2fb29a3700 1 -- 172.21.15.143:0/3664059814 wait complete. 2023-09-29T23:33:54.941 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:55.050 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.046+0000 7f77d7206700 1 Processor -- start 2023-09-29T23:33:55.051 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.046+0000 7f77d7206700 1 -- start start 2023-09-29T23:33:55.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.046+0000 7f77d7206700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f77d00b3140 0x7f77d00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:55.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.046+0000 7f77d7206700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f77d00a8040 0x7f77d013da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:55.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.046+0000 7f77d7206700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d00b3510 con 0x7f77d013d620 2023-09-29T23:33:55.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.046+0000 7f77d7206700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d0141710 con 0x7f77d00b3140 2023-09-29T23:33:55.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.046+0000 7f77d7206700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d01419d0 con 0x7f77d00a8040 2023-09-29T23:33:55.052 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.046+0000 7f77d4fa2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f77d00b3140 0x7f77d00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:55.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.046+0000 7f77d4fa2700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f77d00b3140 0x7f77d00a61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35960/0 (socket says 172.21.15.143:35960) 2023-09-29T23:33:55.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d4fa2700 1 -- 172.21.15.143:0/2253534944 learned_addr learned my addr 172.21.15.143:0/2253534944 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:55.053 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d57a3700 1 --2- 172.21.15.143:0/2253534944 >> v2:172.21.15.156:3300/0 conn(0x7f77d00a8040 0x7f77d013da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:55.054 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d4fa2700 1 -- 172.21.15.143:0/2253534944 >> v2:172.21.15.156:3300/0 conn(0x7f77d00a8040 msgr2=0x7f77d013da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:55.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d4fa2700 1 --2- 172.21.15.143:0/2253534944 >> v2:172.21.15.156:3300/0 conn(0x7f77d00a8040 0x7f77d013da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:55.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d4fa2700 1 -- 172.21.15.143:0/2253534944 >> v1:172.21.15.156:6789/0 conn(0x7f77d013d620 legacy=0x7f77d00a7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:55.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d4fa2700 1 -- 172.21.15.143:0/2253534944 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00a44b0 con 0x7f77d00b3140 2023-09-29T23:33:55.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d4fa2700 1 --2- 172.21.15.143:0/2253534944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d00b3140 0x7f77d00a61c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77d006df90 tx=0x7f77c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2e70291ed519d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:55.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77cf7fe700 1 -- 172.21.15.143:0/2253534944 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4009e70 con 0x7f77d00b3140 2023-09-29T23:33:55.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77cf7fe700 1 -- 172.21.15.143:0/2253534944 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c400b860 con 0x7f77d00b3140 2023-09-29T23:33:55.055 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77cf7fe700 1 -- 172.21.15.143:0/2253534944 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c400bc00 con 0x7f77d00b3140 2023-09-29T23:33:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d7206700 1 -- 172.21.15.143:0/2253534944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d00b3140 msgr2=0x7f77d00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d7206700 1 --2- 172.21.15.143:0/2253534944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d00b3140 0x7f77d00a61c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f77d006df90 tx=0x7f77c4009250 comp rx=0 tx=0).stop 2023-09-29T23:33:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d7206700 1 -- 172.21.15.143:0/2253534944 shutdown_connections 2023-09-29T23:33:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d7206700 1 --2- 172.21.15.143:0/2253534944 >> v2:172.21.15.156:3300/0 conn(0x7f77d00a8040 0x7f77d013da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d7206700 1 --2- 172.21.15.143:0/2253534944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d00b3140 0x7f77d00a61c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d7206700 1 -- 172.21.15.143:0/2253534944 >> 172.21.15.143:0/2253534944 conn(0x7f77d0067dc0 msgr2=0x7f77d00b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d7206700 1 -- 172.21.15.143:0/2253534944 shutdown_connections 2023-09-29T23:33:55.056 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d7206700 1 -- 172.21.15.143:0/2253534944 wait complete. 2023-09-29T23:33:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d7206700 1 Processor -- start 2023-09-29T23:33:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.050+0000 7f77d7206700 1 -- start start 2023-09-29T23:33:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d7206700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d00a8040 0x7f77d011c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d7206700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d00b3140 0x7f77d00aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d7206700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f77d013d620 0x7f77d011aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:55.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d7206700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d011ccb0 con 0x7f77d00a8040 2023-09-29T23:33:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d7206700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d014e4a0 con 0x7f77d00b3140 2023-09-29T23:33:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d7206700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d014e720 con 0x7f77d013d620 2023-09-29T23:33:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d4fa2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d00a8040 0x7f77d011c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d57a3700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f77d013d620 0x7f77d011aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d4fa2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d00a8040 0x7f77d011c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35968/0 (socket says 172.21.15.143:35968) 2023-09-29T23:33:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d4fa2700 1 -- 172.21.15.143:0/1944018657 learned_addr learned my addr 172.21.15.143:0/1944018657 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77cffff700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d00b3140 0x7f77d00aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d4fa2700 1 -- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f77d013d620 msgr2=0x7f77d011aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d4fa2700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f77d013d620 0x7f77d011aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d4fa2700 1 -- 172.21.15.143:0/1944018657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d00b3140 msgr2=0x7f77d00aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:55.061 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d4fa2700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d00b3140 0x7f77d00aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d4fa2700 1 -- 172.21.15.143:0/1944018657 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c40094f0 con 0x7f77d00a8040 2023-09-29T23:33:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d4fa2700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d00a8040 0x7f77d011c770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77d00b3820 tx=0x7f77c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77ccff9700 1 -- 172.21.15.143:0/1944018657 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c400bbe0 con 0x7f77d00a8040 2023-09-29T23:33:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77ccff9700 1 -- 172.21.15.143:0/1944018657 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c4011420 con 0x7f77d00a8040 2023-09-29T23:33:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d7206700 1 -- 172.21.15.143:0/1944018657 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d014e9e0 con 0x7f77d00a8040 2023-09-29T23:33:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77ccff9700 1 -- 172.21.15.143:0/1944018657 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c40117e0 con 0x7f77d00a8040 2023-09-29T23:33:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d7206700 1 -- 172.21.15.143:0/1944018657 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d011a950 con 0x7f77d00a8040 2023-09-29T23:33:55.062 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.054+0000 7f77d7206700 1 -- 172.21.15.143:0/1944018657 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d00b1760 con 0x7f77d00a8040 2023-09-29T23:33:55.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.058+0000 7f77ccff9700 1 -- 172.21.15.143:0/1944018657 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f77c4011bd0 con 0x7f77d00a8040 2023-09-29T23:33:55.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.058+0000 7f77ccff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:55.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.058+0000 7f77ccff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:55.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.058+0000 7f77ccff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:55.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.058+0000 7f77ccff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:55.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.058+0000 7f77ccff9700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f77bc08e8b0 0x7f77bc090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:55.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.058+0000 7f77cffff700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f77bc08e8b0 0x7f77bc090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:55.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.058+0000 7f77ccff9700 1 -- 172.21.15.143:0/1944018657 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f77c40b5d30 con 0x7f77d00a8040 2023-09-29T23:33:55.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.062+0000 7f77cffff700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f77bc08e8b0 0x7f77bc090d60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f77d013d050 tx=0x7f77c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:55.068 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.066+0000 7f77ccff9700 1 -- 172.21.15.143:0/1944018657 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f77d00b1760 con 0x7f77d00a8040 2023-09-29T23:33:55.267 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.262+0000 7f77d7206700 1 -- 172.21.15.143:0/1944018657 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f77d00b1760 con 0x7f77d00a8040 2023-09-29T23:33:55.268 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77ccff9700 1 -- 172.21.15.143:0/1944018657 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f77d00b1760 con 0x7f77d00a8040 2023-09-29T23:33:55.268 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:55.268 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:55.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 10 mgrc shutdown 2023-09-29T23:33:55.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 -- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f77bc08e8b0 msgr2=0x7f77bc090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:55.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f77bc08e8b0 0x7f77bc090d60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f77d013d050 tx=0x7f77c0009040 comp rx=0 tx=0).stop 2023-09-29T23:33:55.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 -- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d00a8040 msgr2=0x7f77d011c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:55.271 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d00a8040 0x7f77d011c770 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f77d00b3820 tx=0x7f77c4017000 comp rx=0 tx=0).stop 2023-09-29T23:33:55.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 -- 172.21.15.143:0/1944018657 shutdown_connections 2023-09-29T23:33:55.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f77bc08e8b0 0x7f77bc090d60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:55.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f77d013d620 0x7f77d011aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:55.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77d00b3140 0x7f77d00aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:55.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 --2- 172.21.15.143:0/1944018657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77d00a8040 0x7f77d011c770 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:55.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 -- 172.21.15.143:0/1944018657 >> 172.21.15.143:0/1944018657 conn(0x7f77d0067dc0 msgr2=0x7f77d00673e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:55.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 -- 172.21.15.143:0/1944018657 shutdown_connections 2023-09-29T23:33:55.272 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:55.266+0000 7f77d7206700 1 -- 172.21.15.143:0/1944018657 wait complete. 2023-09-29T23:33:56.346 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:33:56.349 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:33:58.283 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:33:58.397 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f487359e700 1 Processor -- start 2023-09-29T23:33:58.398 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f487359e700 1 -- start start 2023-09-29T23:33:58.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f487359e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4874066ec0 0x7f487406c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:58.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f487359e700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f487406e930 0x7f48740a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:58.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f487359e700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f487413c010 0x7f487413c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:58.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f487359e700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f487406cd10 con 0x7f487413c010 2023-09-29T23:33:58.399 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f487359e700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4874067290 con 0x7f487406e930 2023-09-29T23:33:58.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f487359e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487406ad80 con 0x7f4874066ec0 2023-09-29T23:33:58.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f4872d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f487413c010 0x7f487413c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:58.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f4872d9d700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f487413c010 0x7f487413c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35980/0 (socket says 172.21.15.143:35980) 2023-09-29T23:33:58.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f4872d9d700 1 -- 172.21.15.143:0/1014291432 learned_addr learned my addr 172.21.15.143:0/1014291432 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:58.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f487259c700 1 --2- 172.21.15.143:0/1014291432 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4874066ec0 0x7f487406c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:58.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f486bfff700 1 --2- 172.21.15.143:0/1014291432 >> v2:172.21.15.156:3300/0 conn(0x7f487406e930 0x7f48740a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:58.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f4872d9d700 1 -- 172.21.15.143:0/1014291432 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4874066ec0 msgr2=0x7f487406c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:58.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f4872d9d700 1 --2- 172.21.15.143:0/1014291432 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4874066ec0 0x7f487406c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f4872d9d700 1 -- 172.21.15.143:0/1014291432 >> v2:172.21.15.156:3300/0 conn(0x7f487406e930 msgr2=0x7f48740a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:33:58.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f4872d9d700 1 --2- 172.21.15.143:0/1014291432 >> v2:172.21.15.156:3300/0 conn(0x7f487406e930 0x7f48740a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.400 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f4872d9d700 1 -- 172.21.15.143:0/1014291432 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487406b040 con 0x7f487413c010 2023-09-29T23:33:58.401 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.394+0000 7f4872d9d700 1 --2- 172.21.15.143:0/1014291432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f487413c010 0x7f487413c490 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4874068060 tx=0x7f486c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf258165721bacb server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:58.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f4871d9b700 1 -- 172.21.15.143:0/1014291432 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c00e070 con 0x7f487413c010 2023-09-29T23:33:58.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f4871d9b700 1 -- 172.21.15.143:0/1014291432 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c00b970 con 0x7f487413c010 2023-09-29T23:33:58.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f4871d9b700 1 -- 172.21.15.143:0/1014291432 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c0092e0 con 0x7f487413c010 2023-09-29T23:33:58.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 -- 172.21.15.143:0/1014291432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f487413c010 msgr2=0x7f487413c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:58.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 --2- 172.21.15.143:0/1014291432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f487413c010 0x7f487413c490 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4874068060 tx=0x7f486c009840 comp rx=0 tx=0).stop 2023-09-29T23:33:58.402 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 -- 172.21.15.143:0/1014291432 shutdown_connections 2023-09-29T23:33:58.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 --2- 172.21.15.143:0/1014291432 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f487413c010 0x7f487413c490 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 --2- 172.21.15.143:0/1014291432 >> v2:172.21.15.156:3300/0 conn(0x7f487406e930 0x7f48740a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 --2- 172.21.15.143:0/1014291432 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f4874066ec0 0x7f487406c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 -- 172.21.15.143:0/1014291432 >> 172.21.15.143:0/1014291432 conn(0x7f487406f200 msgr2=0x7f4874069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:58.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 -- 172.21.15.143:0/1014291432 shutdown_connections 2023-09-29T23:33:58.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 -- 172.21.15.143:0/1014291432 wait complete. 2023-09-29T23:33:58.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 Processor -- start 2023-09-29T23:33:58.403 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 -- start start 2023-09-29T23:33:58.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4874066ec0 0x7f48740a3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:58.406 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f487406e930 0x7f487409a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:58.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487413c010 0x7f4874141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:58.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740a1200 con 0x7f4874066ec0 2023-09-29T23:33:58.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48740a1370 con 0x7f487413c010 2023-09-29T23:33:58.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487359e700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4874142620 con 0x7f487406e930 2023-09-29T23:33:58.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4874066ec0 0x7f48740a3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:58.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487259c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4874066ec0 0x7f48740a3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35996/0 (socket says 172.21.15.143:35996) 2023-09-29T23:33:58.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487259c700 1 -- 172.21.15.143:0/3741753399 learned_addr learned my addr 172.21.15.143:0/3741753399 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:33:58.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f486bfff700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f487406e930 0x7f487409a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:58.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487259c700 1 -- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f487406e930 msgr2=0x7f487409a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:58.407 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487259c700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f487406e930 0x7f487409a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487259c700 1 -- 172.21.15.143:0/3741753399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487413c010 msgr2=0x7f4874141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:58.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487259c700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487413c010 0x7f4874141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487259c700 1 -- 172.21.15.143:0/3741753399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c009ae0 con 0x7f4874066ec0 2023-09-29T23:33:58.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f486bfff700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f487406e930 0x7f487409a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-29T23:33:58.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f487259c700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4874066ec0 0x7f48740a3f60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f486400ec80 tx=0x7f48640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:58.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.398+0000 7f486a7fc700 1 -- 172.21.15.143:0/3741753399 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486400c9d0 con 0x7f4874066ec0 2023-09-29T23:33:58.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.402+0000 7f486a7fc700 1 -- 172.21.15.143:0/3741753399 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486400cb30 con 0x7f4874066ec0 2023-09-29T23:33:58.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.402+0000 7f487359e700 1 -- 172.21.15.143:0/3741753399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48741428e0 con 0x7f4874066ec0 2023-09-29T23:33:58.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.402+0000 7f486a7fc700 1 -- 172.21.15.143:0/3741753399 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4864013470 con 0x7f4874066ec0 2023-09-29T23:33:58.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.402+0000 7f487359e700 1 -- 172.21.15.143:0/3741753399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487409dae0 con 0x7f4874066ec0 2023-09-29T23:33:58.408 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.402+0000 7f487359e700 1 -- 172.21.15.143:0/3741753399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487405ecb0 con 0x7f4874066ec0 2023-09-29T23:33:58.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.406+0000 7f486a7fc700 1 -- 172.21.15.143:0/3741753399 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f4864013820 con 0x7f4874066ec0 2023-09-29T23:33:58.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.406+0000 7f486a7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:33:58.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.406+0000 7f486a7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:33:58.409 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.406+0000 7f486a7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:58.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.406+0000 7f486a7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:33:58.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.406+0000 7f486a7fc700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f485408e8b0 0x7f4854090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:33:58.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.406+0000 7f486bfff700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f485408e8b0 0x7f4854090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:33:58.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.406+0000 7f486a7fc700 1 -- 172.21.15.143:0/3741753399 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f48640b5460 con 0x7f4874066ec0 2023-09-29T23:33:58.410 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.406+0000 7f486bfff700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f485408e8b0 0x7f4854090d60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f485c006f00 tx=0x7f485c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:33:58.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.410+0000 7f486a7fc700 1 -- 172.21.15.143:0/3741753399 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f4864085d80 con 0x7f4874066ec0 2023-09-29T23:33:58.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.606+0000 7f487359e700 1 -- 172.21.15.143:0/3741753399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f487413c9d0 con 0x7f4874066ec0 2023-09-29T23:33:58.610 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.606+0000 7f486a7fc700 1 -- 172.21.15.143:0/3741753399 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f4864021100 con 0x7f4874066ec0 2023-09-29T23:33:58.611 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:33:58.611 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:33:58.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 10 mgrc shutdown 2023-09-29T23:33:58.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 -- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f485408e8b0 msgr2=0x7f4854090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:58.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f485408e8b0 0x7f4854090d60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f485c006f00 tx=0x7f485c008040 comp rx=0 tx=0).stop 2023-09-29T23:33:58.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 -- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4874066ec0 msgr2=0x7f48740a3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:33:58.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4874066ec0 0x7f48740a3f60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f486400ec80 tx=0x7f48640091a0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 -- 172.21.15.143:0/3741753399 shutdown_connections 2023-09-29T23:33:58.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f485408e8b0 0x7f4854090d60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487413c010 0x7f4874141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f487406e930 0x7f487409a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.614 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 --2- 172.21.15.143:0/3741753399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4874066ec0 0x7f48740a3f60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:33:58.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 -- 172.21.15.143:0/3741753399 >> 172.21.15.143:0/3741753399 conn(0x7f487406f200 msgr2=0x7f487413f100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:33:58.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 -- 172.21.15.143:0/3741753399 shutdown_connections 2023-09-29T23:33:58.615 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:33:58.610+0000 7f487359e700 1 -- 172.21.15.143:0/3741753399 wait complete. 2023-09-29T23:34:01.626 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:01.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0c1c8700 1 Processor -- start 2023-09-29T23:34:01.737 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0c1c8700 1 -- start start 2023-09-29T23:34:01.738 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0c1c8700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7feb040a57f0 0x7feb040a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:01.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0c1c8700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb041333c0 0x7feb04140920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:01.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0c1c8700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb040a5bc0 con 0x7feb04140500 2023-09-29T23:34:01.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0c1c8700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb041337f0 con 0x7feb040a57f0 2023-09-29T23:34:01.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0c1c8700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0406d710 con 0x7feb041333c0 2023-09-29T23:34:01.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0a765700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb041333c0 0x7feb04140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:01.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0a765700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb041333c0 0x7feb04140920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:53880/0 (socket says 172.21.15.143:53880) 2023-09-29T23:34:01.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0a765700 1 -- 172.21.15.143:0/3868115117 learned_addr learned my addr 172.21.15.143:0/3868115117 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:01.739 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb09f64700 1 --2- 172.21.15.143:0/3868115117 >> v2:172.21.15.156:3300/0 conn(0x7feb040a57f0 0x7feb040a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:01.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0a765700 1 -- 172.21.15.143:0/3868115117 >> v2:172.21.15.156:3300/0 conn(0x7feb040a57f0 msgr2=0x7feb040a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:01.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0a765700 1 --2- 172.21.15.143:0/3868115117 >> v2:172.21.15.156:3300/0 conn(0x7feb040a57f0 0x7feb040a6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:01.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0a765700 1 -- 172.21.15.143:0/3868115117 >> v1:172.21.15.156:6789/0 conn(0x7feb04140500 legacy=0x7feb04132cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:01.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0a765700 1 -- 172.21.15.143:0/3868115117 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb04141080 con 0x7feb041333c0 2023-09-29T23:34:01.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.734+0000 7feb0a765700 1 --2- 172.21.15.143:0/3868115117 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb041333c0 0x7feb04140920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feafc00d1a0 tx=0x7feafc00ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21902e6ffb71621a server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:01.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb08f62700 1 -- 172.21.15.143:0/3868115117 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc010070 con 0x7feb041333c0 2023-09-29T23:34:01.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb08f62700 1 -- 172.21.15.143:0/3868115117 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00de10 con 0x7feb041333c0 2023-09-29T23:34:01.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb08f62700 1 -- 172.21.15.143:0/3868115117 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00b320 con 0x7feb041333c0 2023-09-29T23:34:01.741 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 -- 172.21.15.143:0/3868115117 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb041333c0 msgr2=0x7feb04140920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:01.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 --2- 172.21.15.143:0/3868115117 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb041333c0 0x7feb04140920 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7feafc00d1a0 tx=0x7feafc00ba50 comp rx=0 tx=0).stop 2023-09-29T23:34:01.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 -- 172.21.15.143:0/3868115117 shutdown_connections 2023-09-29T23:34:01.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 --2- 172.21.15.143:0/3868115117 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb041333c0 0x7feb04140920 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:01.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 --2- 172.21.15.143:0/3868115117 >> v2:172.21.15.156:3300/0 conn(0x7feb040a57f0 0x7feb040a6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:01.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 -- 172.21.15.143:0/3868115117 >> 172.21.15.143:0/3868115117 conn(0x7feb04067dc0 msgr2=0x7feb040b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:01.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 -- 172.21.15.143:0/3868115117 shutdown_connections 2023-09-29T23:34:01.742 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 -- 172.21.15.143:0/3868115117 wait complete. 2023-09-29T23:34:01.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 Processor -- start 2023-09-29T23:34:01.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 -- start start 2023-09-29T23:34:01.744 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb040a57f0 0x7feb040ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb041333c0 0x7feb04141a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb04140500 0x7feb040b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb040ac630 con 0x7feb040a57f0 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb040ab3e0 con 0x7feb041333c0 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0c1c8700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb040a7bc0 con 0x7feb04140500 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0a765700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb04140500 0x7feb040b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb09f64700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb040a57f0 0x7feb040ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0a765700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb04140500 0x7feb040b2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:53888/0 (socket says 172.21.15.143:53888) 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb09f64700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb040a57f0 0x7feb040ac0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36000/0 (socket says 172.21.15.143:36000) 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0a765700 1 -- 172.21.15.143:0/383494404 learned_addr learned my addr 172.21.15.143:0/383494404 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb09763700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb041333c0 0x7feb04141a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:01.745 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0a765700 1 -- 172.21.15.143:0/383494404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb041333c0 msgr2=0x7feb04141a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:01.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0a765700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb041333c0 0x7feb04141a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:01.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0a765700 1 -- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb040a57f0 msgr2=0x7feb040ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:01.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0a765700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb040a57f0 0x7feb040ac0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:01.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0a765700 1 -- 172.21.15.143:0/383494404 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc00bcf0 con 0x7feb04140500 2023-09-29T23:34:01.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb09763700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb041333c0 0x7feb04141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:34:01.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.738+0000 7feb0a765700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb04140500 0x7feb040b2530 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feafc00d840 tx=0x7feafc01a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:01.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feaf9ffb700 1 -- 172.21.15.143:0/383494404 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00ae70 con 0x7feb04140500 2023-09-29T23:34:01.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feaf9ffb700 1 -- 172.21.15.143:0/383494404 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc028460 con 0x7feb04140500 2023-09-29T23:34:01.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feb0c1c8700 1 -- 172.21.15.143:0/383494404 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb040a7e80 con 0x7feb04140500 2023-09-29T23:34:01.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feaf9ffb700 1 -- 172.21.15.143:0/383494404 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc028820 con 0x7feb04140500 2023-09-29T23:34:01.746 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feb0c1c8700 1 -- 172.21.15.143:0/383494404 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb04141540 con 0x7feb04140500 2023-09-29T23:34:01.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feb0c1c8700 1 -- 172.21.15.143:0/383494404 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb04141730 con 0x7feb04140500 2023-09-29T23:34:01.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feaf9ffb700 1 -- 172.21.15.143:0/383494404 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7feafc003040 con 0x7feb04140500 2023-09-29T23:34:01.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feaf9ffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:01.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feaf9ffb700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:01.747 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feaf9ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:01.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feaf9ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:01.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.742+0000 7feaf9ffb700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7feae808e830 0x7feae8090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:01.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.746+0000 7feaf9ffb700 1 -- 172.21.15.143:0/383494404 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7feafc0b8a60 con 0x7feb04140500 2023-09-29T23:34:01.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.746+0000 7feb09f64700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7feae808e830 0x7feae8090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:01.748 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.746+0000 7feb09f64700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7feae808e830 0x7feae8090ce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009db0 tx=0x7feaf40091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:01.753 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.750+0000 7feaf9ffb700 1 -- 172.21.15.143:0/383494404 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7feafc0894b0 con 0x7feb04140500 2023-09-29T23:34:01.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.938+0000 7feb0c1c8700 1 -- 172.21.15.143:0/383494404 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7feb0405ecb0 con 0x7feb04140500 2023-09-29T23:34:01.943 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.938+0000 7feaf9ffb700 1 -- 172.21.15.143:0/383494404 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7feafc023100 con 0x7feb04140500 2023-09-29T23:34:01.943 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:01.944 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:01.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 10 mgrc shutdown 2023-09-29T23:34:01.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 -- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7feae808e830 msgr2=0x7feae8090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:01.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7feae808e830 0x7feae8090ce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009db0 tx=0x7feaf40091a0 comp rx=0 tx=0).stop 2023-09-29T23:34:01.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 -- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb04140500 msgr2=0x7feb040b2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:01.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb04140500 0x7feb040b2530 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feafc00d840 tx=0x7feafc01a000 comp rx=0 tx=0).stop 2023-09-29T23:34:01.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 -- 172.21.15.143:0/383494404 shutdown_connections 2023-09-29T23:34:01.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7feae808e830 0x7feae8090ce0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:01.946 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7feb04140500 0x7feb040b2530 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:01.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb041333c0 0x7feb04141a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:01.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 --2- 172.21.15.143:0/383494404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb040a57f0 0x7feb040ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:01.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 -- 172.21.15.143:0/383494404 >> 172.21.15.143:0/383494404 conn(0x7feb04067dc0 msgr2=0x7feb0406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:01.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 -- 172.21.15.143:0/383494404 shutdown_connections 2023-09-29T23:34:01.947 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:01.942+0000 7feb0c1c8700 1 -- 172.21.15.143:0/383494404 wait complete. 2023-09-29T23:34:04.958 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:05.073 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.070+0000 7f0e39f8d700 1 Processor -- start 2023-09-29T23:34:05.074 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.070+0000 7f0e39f8d700 1 -- start start 2023-09-29T23:34:05.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.070+0000 7f0e39f8d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e3409bcd0 0x7f0e34147c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:05.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.070+0000 7f0e39f8d700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e340a2160 con 0x7f0e340a0550 2023-09-29T23:34:05.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.070+0000 7f0e39f8d700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e340a3c60 con 0x7f0e340a0180 2023-09-29T23:34:05.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.070+0000 7f0e39f8d700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340a3f20 con 0x7f0e3409bcd0 2023-09-29T23:34:05.075 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.070+0000 7f0e3978c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e3409bcd0 0x7f0e34147c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.070+0000 7f0e33fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f0e340a0550 0x7f0e340a1a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34464/0 (socket says 172.21.15.143:34464) 2023-09-29T23:34:05.076 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.070+0000 7f0e33fff700 1 -- 172.21.15.143:0/221384510 learned_addr learned my addr 172.21.15.143:0/221384510 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:34:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e3978c700 1 -- 172.21.15.143:0/221384510 >> v1:172.21.15.156:6789/0 conn(0x7f0e340a0180 legacy=0x7f0e340a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e337fe700 1 -- 172.21.15.143:0/221384510 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243301354 0 0) 0x7f0e340a2160 con 0x7f0e340a0550 2023-09-29T23:34:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e3978c700 1 -- 172.21.15.143:0/221384510 >> v1:172.21.15.143:6789/0 conn(0x7f0e340a0550 legacy=0x7f0e340a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e3978c700 1 -- 172.21.15.143:0/221384510 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e340a3890 con 0x7f0e3409bcd0 2023-09-29T23:34:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e3978c700 1 --2- 172.21.15.143:0/221384510 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e3409bcd0 0x7f0e34147c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009e90 tx=0x7f0e2c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d13b9951fac9c33 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:05.077 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e337fe700 1 -- 172.21.15.143:0/221384510 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c00bc30 con 0x7f0e3409bcd0 2023-09-29T23:34:05.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e337fe700 1 -- 172.21.15.143:0/221384510 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2c00bd90 con 0x7f0e3409bcd0 2023-09-29T23:34:05.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e337fe700 1 -- 172.21.15.143:0/221384510 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c011650 con 0x7f0e3409bcd0 2023-09-29T23:34:05.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 -- 172.21.15.143:0/221384510 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e3409bcd0 msgr2=0x7f0e34147c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:05.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 --2- 172.21.15.143:0/221384510 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e3409bcd0 0x7f0e34147c00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009e90 tx=0x7f0e2c009780 comp rx=0 tx=0).stop 2023-09-29T23:34:05.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 -- 172.21.15.143:0/221384510 shutdown_connections 2023-09-29T23:34:05.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 --2- 172.21.15.143:0/221384510 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e3409bcd0 0x7f0e34147c00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:05.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 -- 172.21.15.143:0/221384510 >> 172.21.15.143:0/221384510 conn(0x7f0e3406a470 msgr2=0x7f0e3406a870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:05.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 -- 172.21.15.143:0/221384510 shutdown_connections 2023-09-29T23:34:05.078 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 -- 172.21.15.143:0/221384510 wait complete. 2023-09-29T23:34:05.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 Processor -- start 2023-09-29T23:34:05.080 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 -- start start 2023-09-29T23:34:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3409bcd0 0x7f0e34144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340a0180 0x7f0e340974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e340a0550 0x7f0e34097a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e34096360 con 0x7f0e340a0180 2023-09-29T23:34:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e340967b0 con 0x7f0e3409bcd0 2023-09-29T23:34:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e39f8d700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e34151060 con 0x7f0e340a0550 2023-09-29T23:34:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e3978c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e340a0550 0x7f0e34097a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e33fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340a0180 0x7f0e340974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e3978c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e340a0550 0x7f0e34097a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:38190/0 (socket says 172.21.15.143:38190) 2023-09-29T23:34:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e3978c700 1 -- 172.21.15.143:0/492875309 learned_addr learned my addr 172.21.15.143:0/492875309 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:05.081 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e38f8b700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3409bcd0 0x7f0e34144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e33fff700 1 -- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e340a0550 msgr2=0x7f0e34097a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e33fff700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e340a0550 0x7f0e34097a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e33fff700 1 -- 172.21.15.143:0/492875309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3409bcd0 msgr2=0x7f0e34144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e33fff700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3409bcd0 0x7f0e34144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e33fff700 1 -- 172.21.15.143:0/492875309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c009a20 con 0x7f0e340a0180 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.074+0000 7f0e38f8b700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3409bcd0 0x7f0e34144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.078+0000 7f0e33fff700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340a0180 0x7f0e340974c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e28000c00 tx=0x7f0e28007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.078+0000 7f0e30ff9700 1 -- 172.21.15.143:0/492875309 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28005b30 con 0x7f0e340a0180 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.078+0000 7f0e30ff9700 1 -- 172.21.15.143:0/492875309 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e28005cb0 con 0x7f0e340a0180 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.078+0000 7f0e30ff9700 1 -- 172.21.15.143:0/492875309 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e28007320 con 0x7f0e340a0180 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.078+0000 7f0e39f8d700 1 -- 172.21.15.143:0/492875309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3413e830 con 0x7f0e340a0180 2023-09-29T23:34:05.082 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.078+0000 7f0e39f8d700 1 -- 172.21.15.143:0/492875309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3413eac0 con 0x7f0e340a0180 2023-09-29T23:34:05.083 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.078+0000 7f0e39f8d700 1 -- 172.21.15.143:0/492875309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e340964d0 con 0x7f0e340a0180 2023-09-29T23:34:05.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.082+0000 7f0e30ff9700 1 -- 172.21.15.143:0/492875309 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f0e28012400 con 0x7f0e340a0180 2023-09-29T23:34:05.084 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.082+0000 7f0e30ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:05.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.082+0000 7f0e30ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:05.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.082+0000 7f0e30ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:05.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.082+0000 7f0e30ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:05.085 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.082+0000 7f0e30ff9700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e2008e8b0 0x7f0e20090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:05.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.082+0000 7f0e38f8b700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e2008e8b0 0x7f0e20090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:05.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.082+0000 7f0e30ff9700 1 -- 172.21.15.143:0/492875309 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f0e280b3c90 con 0x7f0e340a0180 2023-09-29T23:34:05.086 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.082+0000 7f0e38f8b700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e2008e8b0 0x7f0e20090d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0e24007fd0 tx=0x7f0e24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:05.090 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.086+0000 7f0e30ff9700 1 -- 172.21.15.143:0/492875309 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f0e28049be0 con 0x7f0e340a0180 2023-09-29T23:34:05.284 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 -- 172.21.15.143:0/492875309 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0e3405ecb0 con 0x7f0e340a0180 2023-09-29T23:34:05.285 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e30ff9700 1 -- 172.21.15.143:0/492875309 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f0e28017100 con 0x7f0e340a0180 2023-09-29T23:34:05.285 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:05.285 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:05.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 10 mgrc shutdown 2023-09-29T23:34:05.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 -- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e2008e8b0 msgr2=0x7f0e20090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:05.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e2008e8b0 0x7f0e20090d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0e24007fd0 tx=0x7f0e24009040 comp rx=0 tx=0).stop 2023-09-29T23:34:05.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 -- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340a0180 msgr2=0x7f0e340974c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:05.288 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340a0180 0x7f0e340974c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0e28000c00 tx=0x7f0e28007d10 comp rx=0 tx=0).stop 2023-09-29T23:34:05.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 -- 172.21.15.143:0/492875309 shutdown_connections 2023-09-29T23:34:05.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f0e2008e8b0 0x7f0e20090d60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:05.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f0e340a0550 0x7f0e34097a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:05.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e340a0180 0x7f0e340974c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:05.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 --2- 172.21.15.143:0/492875309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3409bcd0 0x7f0e34144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:05.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 -- 172.21.15.143:0/492875309 >> 172.21.15.143:0/492875309 conn(0x7f0e3406a470 msgr2=0x7f0e3406f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:05.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.282+0000 7f0e39f8d700 1 -- 172.21.15.143:0/492875309 shutdown_connections 2023-09-29T23:34:05.289 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:05.286+0000 7f0e39f8d700 1 -- 172.21.15.143:0/492875309 wait complete. 2023-09-29T23:34:08.301 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:08.412 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d3583700 1 Processor -- start 2023-09-29T23:34:08.413 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d3583700 1 -- start start 2023-09-29T23:34:08.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d3583700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12cc13fce0 0x7f12cc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:08.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d3583700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc0a48e0 con 0x7f12cc0b71e0 2023-09-29T23:34:08.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d3583700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc0a3de0 con 0x7f12cc0a5310 2023-09-29T23:34:08.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d3583700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc0a40a0 con 0x7f12cc13fce0 2023-09-29T23:34:08.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d2d82700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f12cc0b71e0 0x7f12cc14aa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34474/0 (socket says 172.21.15.143:34474) 2023-09-29T23:34:08.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d2d82700 1 -- 172.21.15.143:0/3370323706 learned_addr learned my addr 172.21.15.143:0/3370323706 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:34:08.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d1d80700 1 --2- 172.21.15.143:0/3370323706 >> v2:172.21.15.156:3300/0 conn(0x7f12cc13fce0 0x7f12cc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:08.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078453091 0 0) 0x7f12cc0a48e0 con 0x7f12cc0b71e0 2023-09-29T23:34:08.414 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12b8003570 con 0x7f12cc0b71e0 2023-09-29T23:34:08.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941596062 0 0) 0x7f12cc0a3de0 con 0x7f12cc0a5310 2023-09-29T23:34:08.417 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12cc0a48e0 con 0x7f12cc0a5310 2023-09-29T23:34:08.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2071063650 0 0) 0x7f12b8003570 con 0x7f12cc0b71e0 2023-09-29T23:34:08.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 >> v2:172.21.15.156:3300/0 conn(0x7f12cc13fce0 msgr2=0x7f12cc0a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:08.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 --2- 172.21.15.143:0/3370323706 >> v2:172.21.15.156:3300/0 conn(0x7f12cc13fce0 0x7f12cc0a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:08.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 >> v1:172.21.15.156:6789/0 conn(0x7f12cc0a5310 legacy=0x7f12cc13f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:08.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 --> v1:172.21.15.143:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0b67e0 con 0x7f12cc0b71e0 2023-09-29T23:34:08.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 <== mon.0 v1:172.21.15.143:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3251153714 0 0) 0x7f12c8001490 con 0x7f12cc0b71e0 2023-09-29T23:34:08.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 >> v1:172.21.15.143:6789/0 conn(0x7f12cc0b71e0 legacy=0x7f12cc14aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:08.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b8003c40 0x7f12b8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:08.418 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 0x7f12b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:08.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b8005210 0x7f12b800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:08.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc0a40a0 con 0x7f12b8005210 2023-09-29T23:34:08.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b800c130 con 0x7f12b8003c40 2023-09-29T23:34:08.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b800c2c0 con 0x7f12b8004810 2023-09-29T23:34:08.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d1d80700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 0x7f12b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:08.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d2d82700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b8005210 0x7f12b800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:08.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d2581700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b8003c40 0x7f12b8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:08.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d1d80700 1 -- 172.21.15.143:0/3370323706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b8003c40 msgr2=0x7f12b8004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:08.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d1d80700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b8003c40 0x7f12b8004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:08.419 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d1d80700 1 -- 172.21.15.143:0/3370323706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b8005210 msgr2=0x7f12b800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:08.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d1d80700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12b8005210 0x7f12b800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:08.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d1d80700 1 -- 172.21.15.143:0/3370323706 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b8003570 con 0x7f12b8004810 2023-09-29T23:34:08.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d2d82700 1 -- 172.21.15.143:0/3370323706 reap_dead start 2023-09-29T23:34:08.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.410+0000 7f12d2581700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b8003c40 0x7f12b8004030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:34:08.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d1d80700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 0x7f12b8004c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12c400e240 tx=0x7f12c400b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94ed8aa61a9f6a4a server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:08.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400eaf0 con 0x7f12b8004810 2023-09-29T23:34:08.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c400f040 con 0x7f12b8004810 2023-09-29T23:34:08.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d157f700 1 -- 172.21.15.143:0/3370323706 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c400eea0 con 0x7f12b8004810 2023-09-29T23:34:08.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 -- 172.21.15.143:0/3370323706 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 msgr2=0x7f12b8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:08.420 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 0x7f12b8004c80 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f12c400e240 tx=0x7f12c400b040 comp rx=0 tx=0).stop 2023-09-29T23:34:08.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 -- 172.21.15.143:0/3370323706 shutdown_connections 2023-09-29T23:34:08.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 --2- 172.21.15.143:0/3370323706 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 0x7f12b8004c80 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:08.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 -- 172.21.15.143:0/3370323706 >> 172.21.15.143:0/3370323706 conn(0x7f12cc066c90 msgr2=0x7f12cc067090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:08.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 -- 172.21.15.143:0/3370323706 shutdown_connections 2023-09-29T23:34:08.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 -- 172.21.15.143:0/3370323706 wait complete. 2023-09-29T23:34:08.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 Processor -- start 2023-09-29T23:34:08.421 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 -- start start 2023-09-29T23:34:08.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 0x7f12cc1419a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:08.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12cc1428d0 0x7f12cc142d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:08.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc06b3d0 0x7f12cc06b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:08.423 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc142000 con 0x7f12cc1428d0 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc142490 con 0x7f12cc06b3d0 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d3583700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc06bd80 con 0x7f12b8004810 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d2581700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 0x7f12cc1419a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d2581700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 0x7f12cc1419a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:38212/0 (socket says 172.21.15.143:38212) 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d2581700 1 -- 172.21.15.143:0/3139685305 learned_addr learned my addr 172.21.15.143:0/3139685305 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d1d80700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12cc1428d0 0x7f12cc142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d2d82700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc06b3d0 0x7f12cc06b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d2581700 1 -- 172.21.15.143:0/3139685305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc06b3d0 msgr2=0x7f12cc06b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d2581700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc06b3d0 0x7f12cc06b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d2581700 1 -- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12cc1428d0 msgr2=0x7f12cc142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d2581700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12cc1428d0 0x7f12cc142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:08.424 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d2581700 1 -- 172.21.15.143:0/3139685305 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc1433a0 con 0x7f12b8004810 2023-09-29T23:34:08.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12d2581700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 0x7f12cc1419a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00e840 tx=0x7f12bc009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:08.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.414+0000 7f12c27fc700 1 -- 172.21.15.143:0/3139685305 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc006820 con 0x7f12b8004810 2023-09-29T23:34:08.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.418+0000 7f12c27fc700 1 -- 172.21.15.143:0/3139685305 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc006980 con 0x7f12b8004810 2023-09-29T23:34:08.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.418+0000 7f12d3583700 1 -- 172.21.15.143:0/3139685305 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12cc143660 con 0x7f12b8004810 2023-09-29T23:34:08.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.418+0000 7f12c27fc700 1 -- 172.21.15.143:0/3139685305 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc006d00 con 0x7f12b8004810 2023-09-29T23:34:08.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.418+0000 7f12d3583700 1 -- 172.21.15.143:0/3139685305 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc13a810 con 0x7f12b8004810 2023-09-29T23:34:08.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.418+0000 7f12d3583700 1 -- 172.21.15.143:0/3139685305 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc05ecb0 con 0x7f12b8004810 2023-09-29T23:34:08.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.418+0000 7f12c27fc700 1 -- 172.21.15.143:0/3139685305 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f12bc007240 con 0x7f12b8004810 2023-09-29T23:34:08.425 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.418+0000 7f12c27fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:08.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.422+0000 7f12c27fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:08.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.422+0000 7f12c27fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:08.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.422+0000 7f12c27fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:08.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.422+0000 7f12c27fc700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12b008e8e0 0x7f12b0090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:08.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.422+0000 7f12d1d80700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12b008e8e0 0x7f12b0090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:08.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.422+0000 7f12c27fc700 1 -- 172.21.15.143:0/3139685305 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f12bc0b4610 con 0x7f12b8004810 2023-09-29T23:34:08.426 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.422+0000 7f12d1d80700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12b008e8e0 0x7f12b0090d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009c30 tx=0x7f12c401a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:08.435 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.430+0000 7f12c27fc700 1 -- 172.21.15.143:0/3139685305 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f12bc084fd0 con 0x7f12b8004810 2023-09-29T23:34:08.631 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.626+0000 7f12d3583700 1 -- 172.21.15.143:0/3139685305 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f12cc1426b0 con 0x7f12b8004810 2023-09-29T23:34:08.632 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12c27fc700 1 -- 172.21.15.143:0/3139685305 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f12bc019020 con 0x7f12b8004810 2023-09-29T23:34:08.632 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:08.632 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:08.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 10 mgrc shutdown 2023-09-29T23:34:08.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 -- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12b008e8e0 msgr2=0x7f12b0090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:08.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12b008e8e0 0x7f12b0090d90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f12c4009c30 tx=0x7f12c401a000 comp rx=0 tx=0).stop 2023-09-29T23:34:08.634 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 -- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 msgr2=0x7f12cc1419a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 0x7f12cc1419a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f12bc00e840 tx=0x7f12bc009040 comp rx=0 tx=0).stop 2023-09-29T23:34:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 -- 172.21.15.143:0/3139685305 shutdown_connections 2023-09-29T23:34:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f12b008e8e0 0x7f12b0090d90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12cc06b3d0 0x7f12cc06b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f12cc1428d0 0x7f12cc142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 --2- 172.21.15.143:0/3139685305 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f12b8004810 0x7f12cc1419a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 -- 172.21.15.143:0/3139685305 >> 172.21.15.143:0/3139685305 conn(0x7f12cc066c90 msgr2=0x7f12cc0b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 -- 172.21.15.143:0/3139685305 shutdown_connections 2023-09-29T23:34:08.635 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:08.630+0000 7f12d3583700 1 -- 172.21.15.143:0/3139685305 wait complete. 2023-09-29T23:34:11.646 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:11.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.754+0000 7fe80bfff700 1 Processor -- start 2023-09-29T23:34:11.759 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.754+0000 7fe80bfff700 1 -- start start 2023-09-29T23:34:11.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.754+0000 7fe80bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c13d850 0x7fe80c1495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:11.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.754+0000 7fe80bfff700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80c09ff40 con 0x7fe80c066ec0 2023-09-29T23:34:11.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.754+0000 7fe80bfff700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80c067290 con 0x7fe80c0a1170 2023-09-29T23:34:11.760 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.754+0000 7fe80bfff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c0a0510 con 0x7fe80c13d850 2023-09-29T23:34:11.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80b7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c13d850 0x7fe80c1495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:11.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80affd700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fe80c066ec0 0x7fe80c09f830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34476/0 (socket says 172.21.15.143:34476) 2023-09-29T23:34:11.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80affd700 1 -- 172.21.15.143:0/2420851938 learned_addr learned my addr 172.21.15.143:0/2420851938 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:34:11.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe809ffb700 1 -- 172.21.15.143:0/2420851938 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635156812 0 0) 0x7fe80c09ff40 con 0x7fe80c066ec0 2023-09-29T23:34:11.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe809ffb700 1 -- 172.21.15.143:0/2420851938 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7f4003570 con 0x7fe80c066ec0 2023-09-29T23:34:11.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe809ffb700 1 -- 172.21.15.143:0/2420851938 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147664265 0 0) 0x7fe80c067290 con 0x7fe80c0a1170 2023-09-29T23:34:11.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80b7fe700 1 -- 172.21.15.143:0/2420851938 >> v1:172.21.15.156:6789/0 conn(0x7fe80c0a1170 legacy=0x7fe80c13d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:11.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80b7fe700 1 -- 172.21.15.143:0/2420851938 >> v1:172.21.15.143:6789/0 conn(0x7fe80c066ec0 legacy=0x7fe80c09f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:11.761 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80b7fe700 1 -- 172.21.15.143:0/2420851938 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80c06e400 con 0x7fe80c13d850 2023-09-29T23:34:11.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80b7fe700 1 --2- 172.21.15.143:0/2420851938 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c13d850 0x7fe80c1495a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe804009910 tx=0x7fe8040091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7e5a2180a56e2f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:11.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe809ffb700 1 -- 172.21.15.143:0/2420851938 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80400b680 con 0x7fe80c13d850 2023-09-29T23:34:11.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe809ffb700 1 -- 172.21.15.143:0/2420851938 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80400b7e0 con 0x7fe80c13d850 2023-09-29T23:34:11.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe809ffb700 1 -- 172.21.15.143:0/2420851938 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80400bba0 con 0x7fe80c13d850 2023-09-29T23:34:11.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 -- 172.21.15.143:0/2420851938 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c13d850 msgr2=0x7fe80c1495a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:11.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 --2- 172.21.15.143:0/2420851938 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c13d850 0x7fe80c1495a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe804009910 tx=0x7fe8040091a0 comp rx=0 tx=0).stop 2023-09-29T23:34:11.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 -- 172.21.15.143:0/2420851938 shutdown_connections 2023-09-29T23:34:11.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 --2- 172.21.15.143:0/2420851938 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c13d850 0x7fe80c1495a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:11.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 -- 172.21.15.143:0/2420851938 >> 172.21.15.143:0/2420851938 conn(0x7fe80c06f200 msgr2=0x7fe80c069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:11.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 -- 172.21.15.143:0/2420851938 shutdown_connections 2023-09-29T23:34:11.763 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 -- 172.21.15.143:0/2420851938 wait complete. 2023-09-29T23:34:11.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 Processor -- start 2023-09-29T23:34:11.764 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 -- start start 2023-09-29T23:34:11.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe80c066ec0 0x7fe80c139860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:11.765 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c0a1170 0x7fe80c1454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:11.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe80c13d850 0x7fe80c145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:11.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c1447c0 con 0x7fe80c13d850 2023-09-29T23:34:11.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c13a230 con 0x7fe80c066ec0 2023-09-29T23:34:11.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80bfff700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80c13a4f0 con 0x7fe80c0a1170 2023-09-29T23:34:11.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.758+0000 7fe80b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe80c13d850 0x7fe80c145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:11.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80a7fc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c0a1170 0x7fe80c1454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:11.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe80c13d850 0x7fe80c145a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:54386/0 (socket says 172.21.15.143:54386) 2023-09-29T23:34:11.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80b7fe700 1 -- 172.21.15.143:0/1857270131 learned_addr learned my addr 172.21.15.143:0/1857270131 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:11.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80affd700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe80c066ec0 0x7fe80c139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:11.766 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80a7fc700 1 -- 172.21.15.143:0/1857270131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe80c066ec0 msgr2=0x7fe80c139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:11.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80a7fc700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe80c066ec0 0x7fe80c139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:11.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80a7fc700 1 -- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe80c13d850 msgr2=0x7fe80c145a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:11.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80a7fc700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe80c13d850 0x7fe80c145a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:11.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80a7fc700 1 -- 172.21.15.143:0/1857270131 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe804009420 con 0x7fe80c0a1170 2023-09-29T23:34:11.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80b7fe700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe80c13d850 0x7fe80c145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:34:11.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80a7fc700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c0a1170 0x7fe80c1454f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc002230 tx=0x7fe7fc0016f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:11.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe7f2ffd700 1 -- 172.21.15.143:0/1857270131 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc012630 con 0x7fe80c0a1170 2023-09-29T23:34:11.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe7f2ffd700 1 -- 172.21.15.143:0/1857270131 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc0127b0 con 0x7fe80c0a1170 2023-09-29T23:34:11.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe7f2ffd700 1 -- 172.21.15.143:0/1857270131 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc012b70 con 0x7fe80c0a1170 2023-09-29T23:34:11.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80bfff700 1 -- 172.21.15.143:0/1857270131 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80c13a7b0 con 0x7fe80c0a1170 2023-09-29T23:34:11.767 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80bfff700 1 -- 172.21.15.143:0/1857270131 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80c139da0 con 0x7fe80c0a1170 2023-09-29T23:34:11.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.762+0000 7fe80bfff700 1 -- 172.21.15.143:0/1857270131 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80c139ff0 con 0x7fe80c0a1170 2023-09-29T23:34:11.768 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.766+0000 7fe7f2ffd700 1 -- 172.21.15.143:0/1857270131 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fe7fc003030 con 0x7fe80c0a1170 2023-09-29T23:34:11.769 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.766+0000 7fe7f2ffd700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:11.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.766+0000 7fe7f2ffd700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:11.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.766+0000 7fe7f2ffd700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:11.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.766+0000 7fe7f2ffd700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:11.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.766+0000 7fe7f2ffd700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe7f808e8b0 0x7fe7f8090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:11.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.766+0000 7fe80affd700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe7f808e8b0 0x7fe7f8090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:11.770 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.766+0000 7fe7f2ffd700 1 -- 172.21.15.143:0/1857270131 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fe7fc0b4e70 con 0x7fe80c0a1170 2023-09-29T23:34:11.771 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.766+0000 7fe80affd700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe7f808e8b0 0x7fe7f8090d60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe800007fd0 tx=0x7fe800009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:11.773 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.770+0000 7fe7f2ffd700 1 -- 172.21.15.143:0/1857270131 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fe7fc085790 con 0x7fe80c0a1170 2023-09-29T23:34:11.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 -- 172.21.15.143:0/1857270131 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fe80c139ff0 con 0x7fe80c0a1170 2023-09-29T23:34:11.968 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe7f2ffd700 1 -- 172.21.15.143:0/1857270131 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fe80c139ff0 con 0x7fe80c0a1170 2023-09-29T23:34:11.969 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:11.969 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:11.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 10 mgrc shutdown 2023-09-29T23:34:11.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 -- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe7f808e8b0 msgr2=0x7fe7f8090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:11.971 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe7f808e8b0 0x7fe7f8090d60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe800007fd0 tx=0x7fe800009040 comp rx=0 tx=0).stop 2023-09-29T23:34:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 -- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c0a1170 msgr2=0x7fe80c1454f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c0a1170 0x7fe80c1454f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc002230 tx=0x7fe7fc0016f0 comp rx=0 tx=0).stop 2023-09-29T23:34:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 -- 172.21.15.143:0/1857270131 shutdown_connections 2023-09-29T23:34:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fe7f808e8b0 0x7fe7f8090d60 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe80c13d850 0x7fe80c145a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fe80c0a1170 0x7fe80c1454f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 --2- 172.21.15.143:0/1857270131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe80c066ec0 0x7fe80c139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 -- 172.21.15.143:0/1857270131 >> 172.21.15.143:0/1857270131 conn(0x7fe80c06f200 msgr2=0x7fe80c13c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 -- 172.21.15.143:0/1857270131 shutdown_connections 2023-09-29T23:34:11.972 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:11.966+0000 7fe80bfff700 1 -- 172.21.15.143:0/1857270131 wait complete. 2023-09-29T23:34:14.984 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb9105b7700 1 Processor -- start 2023-09-29T23:34:15.096 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb9105b7700 1 -- start start 2023-09-29T23:34:15.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb9105b7700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb9080675d0 0x7fb9080a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:15.097 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb9105b7700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb9080a9530 0x7fb90814ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb9105b7700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9080679a0 con 0x7fb9080a9110 2023-09-29T23:34:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb9105b7700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb90806b4e0 con 0x7fb9080675d0 2023-09-29T23:34:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb9105b7700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90806b7a0 con 0x7fb9080a9530 2023-09-29T23:34:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90e353700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb9080675d0 0x7fb9080a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90e353700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb9080675d0 0x7fb9080a88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37320/0 (socket says 172.21.15.143:37320) 2023-09-29T23:34:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90e353700 1 -- 172.21.15.143:0/2996470150 learned_addr learned my addr 172.21.15.143:0/2996470150 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:15.098 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90eb54700 1 --2- 172.21.15.143:0/2996470150 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb9080a9530 0x7fb90814ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:15.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90e353700 1 -- 172.21.15.143:0/2996470150 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb9080a9530 msgr2=0x7fb90814ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:15.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90e353700 1 --2- 172.21.15.143:0/2996470150 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb9080a9530 0x7fb90814ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:15.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90e353700 1 -- 172.21.15.143:0/2996470150 >> v1:172.21.15.156:6789/0 conn(0x7fb9080a9110 legacy=0x7fb9081361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:15.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90e353700 1 -- 172.21.15.143:0/2996470150 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9080aa5d0 con 0x7fb9080675d0 2023-09-29T23:34:15.099 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90e353700 1 --2- 172.21.15.143:0/2996470150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9080675d0 0x7fb9080a88c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80099b0 tx=0x7fb8f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349c44ef2c72f359 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:15.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90d351700 1 -- 172.21.15.143:0/2996470150 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8009ea0 con 0x7fb9080675d0 2023-09-29T23:34:15.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90d351700 1 -- 172.21.15.143:0/2996470150 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f800b860 con 0x7fb9080675d0 2023-09-29T23:34:15.100 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.094+0000 7fb90d351700 1 -- 172.21.15.143:0/2996470150 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f800bc00 con 0x7fb9080675d0 2023-09-29T23:34:15.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.098+0000 7fb9105b7700 1 -- 172.21.15.143:0/2996470150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9080675d0 msgr2=0x7fb9080a88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:15.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.098+0000 7fb9105b7700 1 --2- 172.21.15.143:0/2996470150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9080675d0 0x7fb9080a88c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb8f80099b0 tx=0x7fb8f8009250 comp rx=0 tx=0).stop 2023-09-29T23:34:15.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.098+0000 7fb9105b7700 1 -- 172.21.15.143:0/2996470150 shutdown_connections 2023-09-29T23:34:15.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.098+0000 7fb9105b7700 1 --2- 172.21.15.143:0/2996470150 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb9080a9530 0x7fb90814ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:15.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.098+0000 7fb9105b7700 1 --2- 172.21.15.143:0/2996470150 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9080675d0 0x7fb9080a88c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:15.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.098+0000 7fb9105b7700 1 -- 172.21.15.143:0/2996470150 >> 172.21.15.143:0/2996470150 conn(0x7fb9080b76e0 msgr2=0x7fb908066c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:15.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.098+0000 7fb9105b7700 1 -- 172.21.15.143:0/2996470150 shutdown_connections 2023-09-29T23:34:15.101 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.098+0000 7fb9105b7700 1 -- 172.21.15.143:0/2996470150 wait complete. 2023-09-29T23:34:15.102 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.098+0000 7fb9105b7700 1 Processor -- start 2023-09-29T23:34:15.103 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.098+0000 7fb9105b7700 1 -- start start 2023-09-29T23:34:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb9105b7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9080675d0 0x7fb9081432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb9105b7700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9080a9110 0x7fb9080ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:15.104 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb9105b7700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb9080a9530 0x7fb9080aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb9105b7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9080ade20 con 0x7fb9080675d0 2023-09-29T23:34:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb9105b7700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9080af260 con 0x7fb9080a9110 2023-09-29T23:34:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb9105b7700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9081446a0 con 0x7fb9080a9530 2023-09-29T23:34:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90e353700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9080675d0 0x7fb9081432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90eb54700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb9080a9530 0x7fb9080aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90e353700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9080675d0 0x7fb9081432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37324/0 (socket says 172.21.15.143:37324) 2023-09-29T23:34:15.105 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90e353700 1 -- 172.21.15.143:0/1823884252 learned_addr learned my addr 172.21.15.143:0/1823884252 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:15.106 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90db52700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9080a9110 0x7fb9080ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90db52700 1 -- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb9080a9530 msgr2=0x7fb9080aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90db52700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb9080a9530 0x7fb9080aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90db52700 1 -- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9080675d0 msgr2=0x7fb9081432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:15.108 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90db52700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9080675d0 0x7fb9081432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:15.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90db52700 1 -- 172.21.15.143:0/1823884252 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f80094f0 con 0x7fb9080a9110 2023-09-29T23:34:15.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90e353700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9080675d0 0x7fb9081432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:34:15.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90eb54700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb9080a9530 0x7fb9080aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:34:15.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb90db52700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9080a9110 0x7fb9080ae7e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb90400c910 tx=0x7fb9040016f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:15.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb8fe7fc700 1 -- 172.21.15.143:0/1823884252 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb904012630 con 0x7fb9080a9110 2023-09-29T23:34:15.109 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb8fe7fc700 1 -- 172.21.15.143:0/1823884252 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9040127b0 con 0x7fb9080a9110 2023-09-29T23:34:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb9105b7700 1 -- 172.21.15.143:0/1823884252 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb908153920 con 0x7fb9080a9110 2023-09-29T23:34:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb8fe7fc700 1 -- 172.21.15.143:0/1823884252 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb904012b70 con 0x7fb9080a9110 2023-09-29T23:34:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb9105b7700 1 -- 172.21.15.143:0/1823884252 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb90812f2c0 con 0x7fb9080a9110 2023-09-29T23:34:15.110 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.102+0000 7fb9105b7700 1 -- 172.21.15.143:0/1823884252 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90812f5a0 con 0x7fb9080a9110 2023-09-29T23:34:15.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.106+0000 7fb8fe7fc700 1 -- 172.21.15.143:0/1823884252 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fb904003030 con 0x7fb9080a9110 2023-09-29T23:34:15.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.106+0000 7fb8fe7fc700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:15.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.106+0000 7fb8fe7fc700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:15.112 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.106+0000 7fb8fe7fc700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:15.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.106+0000 7fb8fe7fc700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:15.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.106+0000 7fb8fe7fc700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb8ec08e8b0 0x7fb8ec090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:15.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.106+0000 7fb90e353700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb8ec08e8b0 0x7fb8ec090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:15.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.110+0000 7fb8fe7fc700 1 -- 172.21.15.143:0/1823884252 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fb9040b4f50 con 0x7fb9080a9110 2023-09-29T23:34:15.113 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.110+0000 7fb90e353700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb8ec08e8b0 0x7fb8ec090d60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8003080 tx=0x7fb8f8017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:15.117 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.114+0000 7fb8fe7fc700 1 -- 172.21.15.143:0/1823884252 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb904085900 con 0x7fb9080a9110 2023-09-29T23:34:15.317 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.314+0000 7fb9105b7700 1 -- 172.21.15.143:0/1823884252 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb9080adf60 con 0x7fb9080a9110 2023-09-29T23:34:15.318 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.314+0000 7fb8fe7fc700 1 -- 172.21.15.143:0/1823884252 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fb904088f20 con 0x7fb9080a9110 2023-09-29T23:34:15.318 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:15.318 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 10 mgrc shutdown 2023-09-29T23:34:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 -- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb8ec08e8b0 msgr2=0x7fb8ec090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:15.320 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb8ec08e8b0 0x7fb8ec090d60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb8f8003080 tx=0x7fb8f8017000 comp rx=0 tx=0).stop 2023-09-29T23:34:15.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 -- 172.21.15.143:0/1823884252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9080a9110 msgr2=0x7fb9080ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:15.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9080a9110 0x7fb9080ae7e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb90400c910 tx=0x7fb9040016f0 comp rx=0 tx=0).stop 2023-09-29T23:34:15.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 -- 172.21.15.143:0/1823884252 shutdown_connections 2023-09-29T23:34:15.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb8ec08e8b0 0x7fb8ec090d60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:15.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb9080a9530 0x7fb9080aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:15.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9080a9110 0x7fb9080ae7e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:15.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 --2- 172.21.15.143:0/1823884252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9080675d0 0x7fb9081432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:15.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 -- 172.21.15.143:0/1823884252 >> 172.21.15.143:0/1823884252 conn(0x7fb9080b76e0 msgr2=0x7fb908135740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:15.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 -- 172.21.15.143:0/1823884252 shutdown_connections 2023-09-29T23:34:15.321 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:15.318+0000 7fb9105b7700 1 -- 172.21.15.143:0/1823884252 wait complete. 2023-09-29T23:34:18.333 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:18.446 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 Processor -- start 2023-09-29T23:34:18.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 -- start start 2023-09-29T23:34:18.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fef540a1170 0x7fef5413d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:18.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef5413d680 0x7fef54148590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:18.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef5409ff40 con 0x7fef54066ec0 2023-09-29T23:34:18.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef54067290 con 0x7fef5413d680 2023-09-29T23:34:18.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef540a0510 con 0x7fef540a1170 2023-09-29T23:34:18.447 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef58d90700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef5413d680 0x7fef54148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:18.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef58d90700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fef5413d680 0x7fef54148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37334/0 (socket says 172.21.15.143:37334) 2023-09-29T23:34:18.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef58d90700 1 -- 172.21.15.143:0/2944262690 learned_addr learned my addr 172.21.15.143:0/2944262690 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:18.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef4b7fe700 1 --2- 172.21.15.143:0/2944262690 >> v2:172.21.15.156:3300/0 conn(0x7fef540a1170 0x7fef5413d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:18.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef58d90700 1 -- 172.21.15.143:0/2944262690 >> v2:172.21.15.156:3300/0 conn(0x7fef540a1170 msgr2=0x7fef5413d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:18.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef58d90700 1 --2- 172.21.15.143:0/2944262690 >> v2:172.21.15.156:3300/0 conn(0x7fef540a1170 0x7fef5413d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:18.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef58d90700 1 -- 172.21.15.143:0/2944262690 >> v1:172.21.15.156:6789/0 conn(0x7fef54066ec0 legacy=0x7fef5409f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:18.448 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef58d90700 1 -- 172.21.15.143:0/2944262690 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5406e400 con 0x7fef5413d680 2023-09-29T23:34:18.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef58d90700 1 --2- 172.21.15.143:0/2944262690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef5413d680 0x7fef54148590 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fef54063a90 tx=0x7fef4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf4ed5d54452d89 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:18.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef537fe700 1 -- 172.21.15.143:0/2944262690 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c00e070 con 0x7fef5413d680 2023-09-29T23:34:18.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef537fe700 1 -- 172.21.15.143:0/2944262690 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4c00ba50 con 0x7fef5413d680 2023-09-29T23:34:18.449 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef537fe700 1 -- 172.21.15.143:0/2944262690 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4c0092e0 con 0x7fef5413d680 2023-09-29T23:34:18.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 -- 172.21.15.143:0/2944262690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef5413d680 msgr2=0x7fef54148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:18.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 --2- 172.21.15.143:0/2944262690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef5413d680 0x7fef54148590 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fef54063a90 tx=0x7fef4c009840 comp rx=0 tx=0).stop 2023-09-29T23:34:18.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 -- 172.21.15.143:0/2944262690 shutdown_connections 2023-09-29T23:34:18.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 --2- 172.21.15.143:0/2944262690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef5413d680 0x7fef54148590 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:18.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 --2- 172.21.15.143:0/2944262690 >> v2:172.21.15.156:3300/0 conn(0x7fef540a1170 0x7fef5413d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:18.451 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 -- 172.21.15.143:0/2944262690 >> 172.21.15.143:0/2944262690 conn(0x7fef5406f200 msgr2=0x7fef54069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:18.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 -- 172.21.15.143:0/2944262690 shutdown_connections 2023-09-29T23:34:18.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.442+0000 7fef59591700 1 -- 172.21.15.143:0/2944262690 wait complete. 2023-09-29T23:34:18.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.446+0000 7fef59591700 1 Processor -- start 2023-09-29T23:34:18.452 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.446+0000 7fef59591700 1 -- start start 2023-09-29T23:34:18.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.446+0000 7fef59591700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef54066ec0 0x7fef54097f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:18.455 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.446+0000 7fef59591700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef540a1170 0x7fef54095a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:18.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.446+0000 7fef59591700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5413d680 0x7fef54095f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:18.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.446+0000 7fef59591700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef540964c0 con 0x7fef54066ec0 2023-09-29T23:34:18.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.446+0000 7fef59591700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef54140860 con 0x7fef5413d680 2023-09-29T23:34:18.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.446+0000 7fef59591700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef54096e20 con 0x7fef540a1170 2023-09-29T23:34:18.456 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef53fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef54066ec0 0x7fef54097f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:18.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef53fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef54066ec0 0x7fef54097f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37342/0 (socket says 172.21.15.143:37342) 2023-09-29T23:34:18.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef4b7fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef540a1170 0x7fef54095a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:18.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef53fff700 1 -- 172.21.15.143:0/1988120454 learned_addr learned my addr 172.21.15.143:0/1988120454 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:18.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef58d90700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5413d680 0x7fef54095f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:18.457 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef53fff700 1 -- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef540a1170 msgr2=0x7fef54095a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:18.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef53fff700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef540a1170 0x7fef54095a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:18.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef53fff700 1 -- 172.21.15.143:0/1988120454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5413d680 msgr2=0x7fef54095f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:18.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef53fff700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5413d680 0x7fef54095f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:18.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef53fff700 1 -- 172.21.15.143:0/1988120454 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4c009ae0 con 0x7fef54066ec0 2023-09-29T23:34:18.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef4b7fe700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef540a1170 0x7fef54095a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:34:18.458 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef53fff700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef54066ec0 0x7fef54097f90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fef4400c590 tx=0x7fef4400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:18.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef50ff9700 1 -- 172.21.15.143:0/1988120454 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44011590 con 0x7fef54066ec0 2023-09-29T23:34:18.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef50ff9700 1 -- 172.21.15.143:0/1988120454 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef44011710 con 0x7fef54066ec0 2023-09-29T23:34:18.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef50ff9700 1 -- 172.21.15.143:0/1988120454 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef44011ad0 con 0x7fef54066ec0 2023-09-29T23:34:18.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef59591700 1 -- 172.21.15.143:0/1988120454 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef540966d0 con 0x7fef54066ec0 2023-09-29T23:34:18.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef59591700 1 -- 172.21.15.143:0/1988120454 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef54140fe0 con 0x7fef54066ec0 2023-09-29T23:34:18.459 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.450+0000 7fef59591700 1 -- 172.21.15.143:0/1988120454 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef5405ecb0 con 0x7fef54066ec0 2023-09-29T23:34:18.460 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.454+0000 7fef50ff9700 1 -- 172.21.15.143:0/1988120454 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fef4401b3b0 con 0x7fef54066ec0 2023-09-29T23:34:18.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.454+0000 7fef50ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:18.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.454+0000 7fef50ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:18.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.454+0000 7fef50ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:18.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.454+0000 7fef50ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:18.461 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.454+0000 7fef50ff9700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fef3408e8e0 0x7fef34090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:18.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.454+0000 7fef4b7fe700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fef3408e8e0 0x7fef34090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:18.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.454+0000 7fef50ff9700 1 -- 172.21.15.143:0/1988120454 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fef440b3d20 con 0x7fef54066ec0 2023-09-29T23:34:18.462 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.454+0000 7fef4b7fe700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fef3408e8e0 0x7fef34090d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef540b22d0 tx=0x7fef3c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:18.464 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.462+0000 7fef50ff9700 1 -- 172.21.15.143:0/1988120454 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fef440846e0 con 0x7fef54066ec0 2023-09-29T23:34:18.661 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.658+0000 7fef59591700 1 -- 172.21.15.143:0/1988120454 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fef5406e6c0 con 0x7fef54066ec0 2023-09-29T23:34:18.662 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.658+0000 7fef50ff9700 1 -- 172.21.15.143:0/1988120454 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fef44017070 con 0x7fef54066ec0 2023-09-29T23:34:18.662 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:18.662 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:18.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 10 mgrc shutdown 2023-09-29T23:34:18.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 -- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fef3408e8e0 msgr2=0x7fef34090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:18.664 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fef3408e8e0 0x7fef34090d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fef540b22d0 tx=0x7fef3c008040 comp rx=0 tx=0).stop 2023-09-29T23:34:18.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 -- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef54066ec0 msgr2=0x7fef54097f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:18.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef54066ec0 0x7fef54097f90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fef4400c590 tx=0x7fef4400c8a0 comp rx=0 tx=0).stop 2023-09-29T23:34:18.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 -- 172.21.15.143:0/1988120454 shutdown_connections 2023-09-29T23:34:18.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fef3408e8e0 0x7fef34090d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:18.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef5413d680 0x7fef54095f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:18.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fef540a1170 0x7fef54095a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:18.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 --2- 172.21.15.143:0/1988120454 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fef54066ec0 0x7fef54097f90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:18.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 -- 172.21.15.143:0/1988120454 >> 172.21.15.143:0/1988120454 conn(0x7fef5406f200 msgr2=0x7fef54069960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:18.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 -- 172.21.15.143:0/1988120454 shutdown_connections 2023-09-29T23:34:18.665 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:18.662+0000 7fef59591700 1 -- 172.21.15.143:0/1988120454 wait complete. 2023-09-29T23:34:21.676 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:21.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d6d13700 1 Processor -- start 2023-09-29T23:34:21.785 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d6d13700 1 -- start start 2023-09-29T23:34:21.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d6d13700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f90d00b3140 0x7f90d006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:21.787 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d6d13700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d013f880 0x7f90d0140210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d6d13700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90d013fd80 con 0x7f90d00687c0 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d6d13700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90d0140750 con 0x7f90d00b3140 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d6d13700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d00b3510 con 0x7f90d013f880 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90cffff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d013f880 0x7f90d0140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d4aaf700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f90d00b3140 0x7f90d006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d52b0700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f90d00687c0 0x7f90d0132220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:43038/0 (socket says 172.21.15.143:43038) 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d52b0700 1 -- 172.21.15.143:0/274978773 learned_addr learned my addr 172.21.15.143:0/274978773 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d4aaf700 1 -- 172.21.15.143:0/274978773 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d013f880 msgr2=0x7f90d0140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d4aaf700 1 --2- 172.21.15.143:0/274978773 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d013f880 0x7f90d0140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d4aaf700 1 -- 172.21.15.143:0/274978773 >> v1:172.21.15.143:6789/0 conn(0x7f90d00687c0 legacy=0x7f90d0132220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d4aaf700 1 -- 172.21.15.143:0/274978773 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d006b8c0 con 0x7f90d00b3140 2023-09-29T23:34:21.788 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90cffff700 1 --2- 172.21.15.143:0/274978773 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d013f880 0x7f90d0140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:34:21.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90d4aaf700 1 --2- 172.21.15.143:0/274978773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d00b3140 0x7f90d006d8d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009910 tx=0x7f90c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d8d6ae3a8b6f2f server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:21.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90cf7fe700 1 -- 172.21.15.143:0/274978773 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c400b640 con 0x7f90d00b3140 2023-09-29T23:34:21.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90cf7fe700 1 -- 172.21.15.143:0/274978773 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c400b7a0 con 0x7f90d00b3140 2023-09-29T23:34:21.789 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.782+0000 7f90cf7fe700 1 -- 172.21.15.143:0/274978773 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c400bb60 con 0x7f90d00b3140 2023-09-29T23:34:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.786+0000 7f90d6d13700 1 -- 172.21.15.143:0/274978773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d00b3140 msgr2=0x7f90d006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.786+0000 7f90d6d13700 1 --2- 172.21.15.143:0/274978773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d00b3140 0x7f90d006d8d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009910 tx=0x7f90c40091a0 comp rx=0 tx=0).stop 2023-09-29T23:34:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.786+0000 7f90d6d13700 1 -- 172.21.15.143:0/274978773 shutdown_connections 2023-09-29T23:34:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.786+0000 7f90d6d13700 1 --2- 172.21.15.143:0/274978773 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d013f880 0x7f90d0140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.786+0000 7f90d6d13700 1 --2- 172.21.15.143:0/274978773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d00b3140 0x7f90d006d8d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.786+0000 7f90d6d13700 1 -- 172.21.15.143:0/274978773 >> 172.21.15.143:0/274978773 conn(0x7f90d0067dc0 msgr2=0x7f90d00b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.786+0000 7f90d6d13700 1 -- 172.21.15.143:0/274978773 shutdown_connections 2023-09-29T23:34:21.790 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.786+0000 7f90d6d13700 1 -- 172.21.15.143:0/274978773 wait complete. 2023-09-29T23:34:21.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.786+0000 7f90d6d13700 1 Processor -- start 2023-09-29T23:34:21.791 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.786+0000 7f90d6d13700 1 -- start start 2023-09-29T23:34:21.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d6d13700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d00687c0 0x7f90d012cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:21.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d6d13700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90d013f880 0x7f90d01411a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:21.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d6d13700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d012e8c0 0x7f90d01416e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:21.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d6d13700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d01285c0 con 0x7f90d012e8c0 2023-09-29T23:34:21.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d6d13700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90d012d900 con 0x7f90d013f880 2023-09-29T23:34:21.795 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d6d13700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90d012dbc0 con 0x7f90d00687c0 2023-09-29T23:34:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d4aaf700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d00687c0 0x7f90d012cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d52b0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d012e8c0 0x7f90d01416e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d4aaf700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d00687c0 0x7f90d012cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:43764/0 (socket says 172.21.15.143:43764) 2023-09-29T23:34:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d4aaf700 1 -- 172.21.15.143:0/1270093335 learned_addr learned my addr 172.21.15.143:0/1270093335 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90cffff700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90d013f880 0x7f90d01411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d4aaf700 1 -- 172.21.15.143:0/1270093335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90d013f880 msgr2=0x7f90d01411a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d4aaf700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90d013f880 0x7f90d01411a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d4aaf700 1 -- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d012e8c0 msgr2=0x7f90d01416e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d4aaf700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d012e8c0 0x7f90d01416e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d4aaf700 1 -- 172.21.15.143:0/1270093335 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c0009420 con 0x7f90d00687c0 2023-09-29T23:34:21.796 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90cffff700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90d013f880 0x7f90d01411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:34:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d52b0700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d012e8c0 0x7f90d01416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:34:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d4aaf700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d00687c0 0x7f90d012cdf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009fb0 tx=0x7f90c4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90ccff9700 1 -- 172.21.15.143:0/1270093335 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c401f070 con 0x7f90d00687c0 2023-09-29T23:34:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90ccff9700 1 -- 172.21.15.143:0/1270093335 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c4000bc0 con 0x7f90d00687c0 2023-09-29T23:34:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d6d13700 1 -- 172.21.15.143:0/1270093335 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c4009420 con 0x7f90d00687c0 2023-09-29T23:34:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90ccff9700 1 -- 172.21.15.143:0/1270093335 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c400bb40 con 0x7f90d00687c0 2023-09-29T23:34:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.790+0000 7f90d6d13700 1 -- 172.21.15.143:0/1270093335 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90d012c000 con 0x7f90d00687c0 2023-09-29T23:34:21.797 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.794+0000 7f90d6d13700 1 -- 172.21.15.143:0/1270093335 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90d005ecb0 con 0x7f90d00687c0 2023-09-29T23:34:21.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.794+0000 7f90ccff9700 1 -- 172.21.15.143:0/1270093335 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f90c4023070 con 0x7f90d00687c0 2023-09-29T23:34:21.798 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.794+0000 7f90ccff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:21.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.794+0000 7f90ccff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:21.799 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.794+0000 7f90ccff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:21.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.794+0000 7f90ccff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:21.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.794+0000 7f90ccff9700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f90bc08e8b0 0x7f90bc090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:21.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.794+0000 7f90cffff700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f90bc08e8b0 0x7f90bc090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:21.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.798+0000 7f90ccff9700 1 -- 172.21.15.143:0/1270093335 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f90c40b5af0 con 0x7f90d00687c0 2023-09-29T23:34:21.800 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.798+0000 7f90cffff700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f90bc08e8b0 0x7f90bc090d60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f90c00093f0 tx=0x7f90c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:21.805 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.802+0000 7f90ccff9700 1 -- 172.21.15.143:0/1270093335 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f90c40864a0 con 0x7f90d00687c0 2023-09-29T23:34:21.995 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.990+0000 7f90d6d13700 1 -- 172.21.15.143:0/1270093335 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f90d0132930 con 0x7f90d00687c0 2023-09-29T23:34:21.996 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90ccff9700 1 -- 172.21.15.143:0/1270093335 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f90c4089ac0 con 0x7f90d00687c0 2023-09-29T23:34:21.996 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:21.996 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:21.998 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 10 mgrc shutdown 2023-09-29T23:34:21.999 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 -- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f90bc08e8b0 msgr2=0x7f90bc090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f90bc08e8b0 0x7f90bc090d60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f90c00093f0 tx=0x7f90c0008040 comp rx=0 tx=0).stop 2023-09-29T23:34:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 -- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d00687c0 msgr2=0x7f90d012cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:22.001 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d00687c0 0x7f90d012cdf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f90c4009fb0 tx=0x7f90c4017000 comp rx=0 tx=0).stop 2023-09-29T23:34:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 -- 172.21.15.143:0/1270093335 shutdown_connections 2023-09-29T23:34:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f90bc08e8b0 0x7f90bc090d60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90d012e8c0 0x7f90d01416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:22.002 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90d013f880 0x7f90d01411a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:22.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 --2- 172.21.15.143:0/1270093335 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f90d00687c0 0x7f90d012cdf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:22.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 -- 172.21.15.143:0/1270093335 >> 172.21.15.143:0/1270093335 conn(0x7f90d0067dc0 msgr2=0x7f90d0131ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:22.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 -- 172.21.15.143:0/1270093335 shutdown_connections 2023-09-29T23:34:22.003 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:21.994+0000 7f90d6d13700 1 -- 172.21.15.143:0/1270093335 wait complete. 2023-09-29T23:34:25.014 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:25.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 Processor -- start 2023-09-29T23:34:25.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 -- start start 2023-09-29T23:34:25.124 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 0x7f60e40a6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:25.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f60e40a6670 0x7f60e4148b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:25.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60e41417f0 con 0x7f60e40a7850 2023-09-29T23:34:25.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60e413e120 con 0x7f60e40a6670 2023-09-29T23:34:25.125 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e40a6f90 con 0x7f60e40a83f0 2023-09-29T23:34:25.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e37fe700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f60e40a7850 0x7f60e413da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:33136/0 (socket says 172.21.15.143:33136) 2023-09-29T23:34:25.126 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e37fe700 1 -- 172.21.15.143:0/1221809431 learned_addr learned my addr 172.21.15.143:0/1221809431 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60dbfff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 0x7f60e40a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e3fff700 1 --2- 172.21.15.143:0/1221809431 >> v2:172.21.15.143:3300/0 conn(0x7f60e40a6670 0x7f60e4148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e2ffd700 1 -- 172.21.15.143:0/1221809431 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836683721 0 0) 0x7f60e41417f0 con 0x7f60e40a7850 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e2ffd700 1 -- 172.21.15.143:0/1221809431 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60d0003570 con 0x7f60e40a7850 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e2ffd700 1 -- 172.21.15.143:0/1221809431 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 507173268 0 0) 0x7f60d0003570 con 0x7f60e40a7850 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e3fff700 1 -- 172.21.15.143:0/1221809431 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 msgr2=0x7f60e40a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e3fff700 1 --2- 172.21.15.143:0/1221809431 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 0x7f60e40a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e3fff700 1 -- 172.21.15.143:0/1221809431 >> v1:172.21.15.143:6789/0 conn(0x7f60e40a7850 legacy=0x7f60e413da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e3fff700 1 -- 172.21.15.143:0/1221809431 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60e4141a90 con 0x7f60e40a6670 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60dbfff700 1 --2- 172.21.15.143:0/1221809431 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 0x7f60e40a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e3fff700 1 --2- 172.21.15.143:0/1221809431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60e40a6670 0x7f60e4148b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009e60 tx=0x7f60dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376dbfc52edd00c server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:25.127 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e2ffd700 1 -- 172.21.15.143:0/1221809431 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc00e070 con 0x7f60e40a6670 2023-09-29T23:34:25.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e2ffd700 1 -- 172.21.15.143:0/1221809431 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc00b7a0 con 0x7f60e40a6670 2023-09-29T23:34:25.128 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60e2ffd700 1 -- 172.21.15.143:0/1221809431 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d40015f0 con 0x7f60e40a6670 2023-09-29T23:34:25.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 -- 172.21.15.143:0/1221809431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60e40a6670 msgr2=0x7f60e4148b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:25.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 --2- 172.21.15.143:0/1221809431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60e40a6670 0x7f60e4148b90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009e60 tx=0x7f60dc009780 comp rx=0 tx=0).stop 2023-09-29T23:34:25.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 -- 172.21.15.143:0/1221809431 shutdown_connections 2023-09-29T23:34:25.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 --2- 172.21.15.143:0/1221809431 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60e40a6670 0x7f60e4148b90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:25.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 --2- 172.21.15.143:0/1221809431 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 0x7f60e40a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:25.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 -- 172.21.15.143:0/1221809431 >> 172.21.15.143:0/1221809431 conn(0x7f60e40b5410 msgr2=0x7f60e4066c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:25.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 -- 172.21.15.143:0/1221809431 shutdown_connections 2023-09-29T23:34:25.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.122+0000 7f60ea1bc700 1 -- 172.21.15.143:0/1221809431 wait complete. 2023-09-29T23:34:25.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60ea1bc700 1 Processor -- start 2023-09-29T23:34:25.129 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60ea1bc700 1 -- start start 2023-09-29T23:34:25.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60ea1bc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60e40a6670 0x7f60e4141250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:25.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60ea1bc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60e40a7850 0x7f60e40ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:25.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60ea1bc700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 0x7f60e40ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:25.131 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60ea1bc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e413f5f0 con 0x7f60e40a6670 2023-09-29T23:34:25.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60ea1bc700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e4141790 con 0x7f60e40a7850 2023-09-29T23:34:25.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60ea1bc700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60e406a370 con 0x7f60e40a83f0 2023-09-29T23:34:25.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60e37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60e40a6670 0x7f60e4141250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:25.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60e3fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 0x7f60e40ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:25.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60e37fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60e40a6670 0x7f60e4141250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51090/0 (socket says 172.21.15.143:51090) 2023-09-29T23:34:25.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60e37fe700 1 -- 172.21.15.143:0/3087234071 learned_addr learned my addr 172.21.15.143:0/3087234071 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:25.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60dbfff700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60e40a7850 0x7f60e40ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:25.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60e37fe700 1 -- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 msgr2=0x7f60e40ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:25.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60e37fe700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 0x7f60e40ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:25.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60e37fe700 1 -- 172.21.15.143:0/3087234071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60e40a7850 msgr2=0x7f60e40ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:25.132 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60e37fe700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60e40a7850 0x7f60e40ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:25.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60e37fe700 1 -- 172.21.15.143:0/3087234071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60cc009420 con 0x7f60e40a6670 2023-09-29T23:34:25.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60e3fff700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 0x7f60e40ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:34:25.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60e37fe700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60e40a6670 0x7f60e4141250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f60d400c910 tx=0x7f60d4000f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:25.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60db7fe700 1 -- 172.21.15.143:0/3087234071 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d4007300 con 0x7f60e40a6670 2023-09-29T23:34:25.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60db7fe700 1 -- 172.21.15.143:0/3087234071 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d4007460 con 0x7f60e40a6670 2023-09-29T23:34:25.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60ea1bc700 1 -- 172.21.15.143:0/3087234071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60dc009a20 con 0x7f60e40a6670 2023-09-29T23:34:25.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60db7fe700 1 -- 172.21.15.143:0/3087234071 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d40057f0 con 0x7f60e40a6670 2023-09-29T23:34:25.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.126+0000 7f60ea1bc700 1 -- 172.21.15.143:0/3087234071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e40adb40 con 0x7f60e40a6670 2023-09-29T23:34:25.133 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.130+0000 7f60ea1bc700 1 -- 172.21.15.143:0/3087234071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e40af230 con 0x7f60e40a6670 2023-09-29T23:34:25.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.130+0000 7f60db7fe700 1 -- 172.21.15.143:0/3087234071 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f60d4005c20 con 0x7f60e40a6670 2023-09-29T23:34:25.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.130+0000 7f60db7fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:25.135 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.134+0000 7f60db7fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:25.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.134+0000 7f60db7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:25.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.134+0000 7f60db7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:25.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.134+0000 7f60db7fe700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f60c408e830 0x7f60c4090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:25.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.134+0000 7f60db7fe700 1 -- 172.21.15.143:0/3087234071 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f60d40b45b0 con 0x7f60e40a6670 2023-09-29T23:34:25.136 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.134+0000 7f60dbfff700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f60c408e830 0x7f60c4090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:25.137 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.134+0000 7f60dbfff700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f60c408e830 0x7f60c4090ce0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0093f0 tx=0x7f60cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:25.144 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.142+0000 7f60db7fe700 1 -- 172.21.15.143:0/3087234071 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f60d4085000 con 0x7f60e40a6670 2023-09-29T23:34:25.340 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.338+0000 7f60ea1bc700 1 -- 172.21.15.143:0/3087234071 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f60e405ecb0 con 0x7f60e40a6670 2023-09-29T23:34:25.341 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.338+0000 7f60db7fe700 1 -- 172.21.15.143:0/3087234071 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f60d4088620 con 0x7f60e40a6670 2023-09-29T23:34:25.341 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:25.341 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 10 mgrc shutdown 2023-09-29T23:34:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 -- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f60c408e830 msgr2=0x7f60c4090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:25.344 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f60c408e830 0x7f60c4090ce0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f60cc0093f0 tx=0x7f60cc008040 comp rx=0 tx=0).stop 2023-09-29T23:34:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 -- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60e40a6670 msgr2=0x7f60e4141250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60e40a6670 0x7f60e4141250 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f60d400c910 tx=0x7f60d4000f70 comp rx=0 tx=0).stop 2023-09-29T23:34:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 -- 172.21.15.143:0/3087234071 shutdown_connections 2023-09-29T23:34:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f60c408e830 0x7f60c4090ce0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f60e40a83f0 0x7f60e40ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f60e40a7850 0x7f60e40ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 --2- 172.21.15.143:0/3087234071 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f60e40a6670 0x7f60e4141250 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 -- 172.21.15.143:0/3087234071 >> 172.21.15.143:0/3087234071 conn(0x7f60e40b5410 msgr2=0x7f60e40b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 -- 172.21.15.143:0/3087234071 shutdown_connections 2023-09-29T23:34:25.345 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:25.342+0000 7f60ea1bc700 1 -- 172.21.15.143:0/3087234071 wait complete. 2023-09-29T23:34:26.413 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:34:26.416 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:34:28.362 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:28.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.470+0000 7f66ba3be700 1 Processor -- start 2023-09-29T23:34:28.474 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.470+0000 7f66ba3be700 1 -- start start 2023-09-29T23:34:28.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.470+0000 7f66ba3be700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f66b413a300 0x7f66b413a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:28.475 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.470+0000 7f66ba3be700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66b40a0d60 0x7f66b40a1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:28.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.470+0000 7f66ba3be700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b40a19f0 con 0x7f66b4066ec0 2023-09-29T23:34:28.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.470+0000 7f66ba3be700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b4067290 con 0x7f66b40a0d60 2023-09-29T23:34:28.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.470+0000 7f66ba3be700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b409f560 con 0x7f66b413a300 2023-09-29T23:34:28.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.470+0000 7f66b9bbd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66b40a0d60 0x7f66b40a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:28.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.470+0000 7f66b9bbd700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f66b40a0d60 0x7f66b40a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51100/0 (socket says 172.21.15.143:51100) 2023-09-29T23:34:28.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.470+0000 7f66b9bbd700 1 -- 172.21.15.143:0/3489743167 learned_addr learned my addr 172.21.15.143:0/3489743167 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:28.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.470+0000 7f66b8bbb700 1 --2- 172.21.15.143:0/3489743167 >> v2:172.21.15.156:3300/0 conn(0x7f66b413a300 0x7f66b413a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:28.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66b9bbd700 1 -- 172.21.15.143:0/3489743167 >> v2:172.21.15.156:3300/0 conn(0x7f66b413a300 msgr2=0x7f66b413a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:28.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66b9bbd700 1 --2- 172.21.15.143:0/3489743167 >> v2:172.21.15.156:3300/0 conn(0x7f66b413a300 0x7f66b413a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:28.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66b9bbd700 1 -- 172.21.15.143:0/3489743167 >> v1:172.21.15.156:6789/0 conn(0x7f66b4066ec0 legacy=0x7f66b40aee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:28.476 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66b9bbd700 1 -- 172.21.15.143:0/3489743167 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40a20b0 con 0x7f66b40a0d60 2023-09-29T23:34:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66b9bbd700 1 --2- 172.21.15.143:0/3489743167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b40a0d60 0x7f66b40a1130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66b4068010 tx=0x7f66ac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54dbe9c7903407cf server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66abfff700 1 -- 172.21.15.143:0/3489743167 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac00e070 con 0x7f66b40a0d60 2023-09-29T23:34:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66abfff700 1 -- 172.21.15.143:0/3489743167 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac00bab0 con 0x7f66b40a0d60 2023-09-29T23:34:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66abfff700 1 -- 172.21.15.143:0/3489743167 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0092e0 con 0x7f66b40a0d60 2023-09-29T23:34:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66ba3be700 1 -- 172.21.15.143:0/3489743167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b40a0d60 msgr2=0x7f66b40a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66ba3be700 1 --2- 172.21.15.143:0/3489743167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b40a0d60 0x7f66b40a1130 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f66b4068010 tx=0x7f66ac009840 comp rx=0 tx=0).stop 2023-09-29T23:34:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66ba3be700 1 -- 172.21.15.143:0/3489743167 shutdown_connections 2023-09-29T23:34:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66ba3be700 1 --2- 172.21.15.143:0/3489743167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b40a0d60 0x7f66b40a1130 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:28.478 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66ba3be700 1 --2- 172.21.15.143:0/3489743167 >> v2:172.21.15.156:3300/0 conn(0x7f66b413a300 0x7f66b413a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:28.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66ba3be700 1 -- 172.21.15.143:0/3489743167 >> 172.21.15.143:0/3489743167 conn(0x7f66b406f200 msgr2=0x7f66b4069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:28.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66ba3be700 1 -- 172.21.15.143:0/3489743167 shutdown_connections 2023-09-29T23:34:28.479 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.474+0000 7f66ba3be700 1 -- 172.21.15.143:0/3489743167 wait complete. 2023-09-29T23:34:28.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66ba3be700 1 Processor -- start 2023-09-29T23:34:28.480 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66ba3be700 1 -- start start 2023-09-29T23:34:28.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66ba3be700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b4066ec0 0x7f66b4095c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:28.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66ba3be700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66b40a0d60 0x7f66b406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:28.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66ba3be700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f66b413a300 0x7f66b4157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:28.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66ba3be700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b4096a10 con 0x7f66b4066ec0 2023-09-29T23:34:28.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66ba3be700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40967a0 con 0x7f66b40a0d60 2023-09-29T23:34:28.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66ba3be700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b406dfe0 con 0x7f66b413a300 2023-09-29T23:34:28.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66b93bc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b4066ec0 0x7f66b4095c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:28.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66b93bc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b4066ec0 0x7f66b4095c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51116/0 (socket says 172.21.15.143:51116) 2023-09-29T23:34:28.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66b93bc700 1 -- 172.21.15.143:0/3747081553 learned_addr learned my addr 172.21.15.143:0/3747081553 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:28.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66b9bbd700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f66b413a300 0x7f66b4157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:28.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66b8bbb700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66b40a0d60 0x7f66b406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:28.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66b93bc700 1 -- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f66b413a300 msgr2=0x7f66b4157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:28.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66b93bc700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f66b413a300 0x7f66b4157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:28.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66b93bc700 1 -- 172.21.15.143:0/3747081553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66b40a0d60 msgr2=0x7f66b406d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:28.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66b93bc700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66b40a0d60 0x7f66b406d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66b93bc700 1 -- 172.21.15.143:0/3747081553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac009ae0 con 0x7f66b4066ec0 2023-09-29T23:34:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66b93bc700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b4066ec0 0x7f66b4095c70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66a400c590 tx=0x7f66a400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66a97fa700 1 -- 172.21.15.143:0/3747081553 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4011590 con 0x7f66b4066ec0 2023-09-29T23:34:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66a97fa700 1 -- 172.21.15.143:0/3747081553 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a40116f0 con 0x7f66b4066ec0 2023-09-29T23:34:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66ba3be700 1 -- 172.21.15.143:0/3747081553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b406e2a0 con 0x7f66b4066ec0 2023-09-29T23:34:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66a97fa700 1 -- 172.21.15.143:0/3747081553 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4011ab0 con 0x7f66b4066ec0 2023-09-29T23:34:28.485 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.478+0000 7f66ba3be700 1 -- 172.21.15.143:0/3747081553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b406c990 con 0x7f66b4066ec0 2023-09-29T23:34:28.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.482+0000 7f66ba3be700 1 -- 172.21.15.143:0/3747081553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b4097500 con 0x7f66b4066ec0 2023-09-29T23:34:28.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.482+0000 7f66a97fa700 1 -- 172.21.15.143:0/3747081553 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f66a401b3b0 con 0x7f66b4066ec0 2023-09-29T23:34:28.486 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.482+0000 7f66a97fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:28.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.486+0000 7f66a97fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.486+0000 7f66a97fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.486+0000 7f66a97fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.486+0000 7f66a97fa700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f669808e8b0 0x7f6698090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.486+0000 7f66b8bbb700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f669808e8b0 0x7f6698090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.486+0000 7f66a97fa700 1 -- 172.21.15.143:0/3747081553 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f66a40b3e10 con 0x7f66b4066ec0 2023-09-29T23:34:28.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.486+0000 7f66b8bbb700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f669808e8b0 0x7f6698090d60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009ca0 tx=0x7f66b0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:28.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.490+0000 7f66a97fa700 1 -- 172.21.15.143:0/3747081553 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f66a40847d0 con 0x7f66b4066ec0 2023-09-29T23:34:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.682+0000 7f66ba3be700 1 -- 172.21.15.143:0/3747081553 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f66b405ecb0 con 0x7f66b4066ec0 2023-09-29T23:34:28.685 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.682+0000 7f66a97fa700 1 -- 172.21.15.143:0/3747081553 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f66a4017070 con 0x7f66b4066ec0 2023-09-29T23:34:28.686 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:28.686 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:28.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.682+0000 7f66ba3be700 10 mgrc shutdown 2023-09-29T23:34:28.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.682+0000 7f66ba3be700 1 -- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f669808e8b0 msgr2=0x7f6698090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:28.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.686+0000 7f66ba3be700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f669808e8b0 0x7f6698090d60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f66b0009ca0 tx=0x7f66b0008040 comp rx=0 tx=0).stop 2023-09-29T23:34:28.688 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.686+0000 7f66ba3be700 1 -- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b4066ec0 msgr2=0x7f66b4095c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:28.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.686+0000 7f66ba3be700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b4066ec0 0x7f66b4095c70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f66a400c590 tx=0x7f66a400c950 comp rx=0 tx=0).stop 2023-09-29T23:34:28.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.686+0000 7f66ba3be700 1 -- 172.21.15.143:0/3747081553 shutdown_connections 2023-09-29T23:34:28.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.686+0000 7f66ba3be700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f669808e8b0 0x7f6698090d60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:28.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.686+0000 7f66ba3be700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f66b413a300 0x7f66b4157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:28.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.686+0000 7f66ba3be700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66b40a0d60 0x7f66b406d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:28.689 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.686+0000 7f66ba3be700 1 --2- 172.21.15.143:0/3747081553 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b4066ec0 0x7f66b4095c70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:28.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.686+0000 7f66ba3be700 1 -- 172.21.15.143:0/3747081553 >> 172.21.15.143:0/3747081553 conn(0x7f66b406f200 msgr2=0x7f66b406fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:28.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.686+0000 7f66ba3be700 1 -- 172.21.15.143:0/3747081553 shutdown_connections 2023-09-29T23:34:28.690 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:28.686+0000 7f66ba3be700 1 -- 172.21.15.143:0/3747081553 wait complete. 2023-09-29T23:34:31.700 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:31.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.805+0000 7f90794e9700 1 Processor -- start 2023-09-29T23:34:31.811 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.805+0000 7f90794e9700 1 -- start start 2023-09-29T23:34:31.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.805+0000 7f90794e9700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9074066ec0 0x7f90740aee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:31.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.805+0000 7f90794e9700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 0x7f907413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:31.812 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.805+0000 7f90794e9700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f90740a0d60 0x7f90740a1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:31.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.805+0000 7f90794e9700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90740a19f0 con 0x7f90740a0d60 2023-09-29T23:34:31.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.805+0000 7f90794e9700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9074067290 con 0x7f9074066ec0 2023-09-29T23:34:31.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.805+0000 7f90794e9700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907409f560 con 0x7f907413a300 2023-09-29T23:34:31.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.805+0000 7f90737fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 0x7f907413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:31.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.805+0000 7f9078ce8700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f90740a0d60 0x7f90740a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:31.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.805+0000 7f90737fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 0x7f907413a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:33374/0 (socket says 172.21.15.143:33374) 2023-09-29T23:34:31.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90737fe700 1 -- 172.21.15.143:0/2237727534 learned_addr learned my addr 172.21.15.143:0/2237727534 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:31.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f9073fff700 1 --2- 172.21.15.143:0/2237727534 >> v2:172.21.15.156:3300/0 conn(0x7f9074066ec0 0x7f90740aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:31.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f9073fff700 1 -- 172.21.15.143:0/2237727534 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 msgr2=0x7f907413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:31.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f9073fff700 1 --2- 172.21.15.143:0/2237727534 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 0x7f907413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:31.813 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f9073fff700 1 -- 172.21.15.143:0/2237727534 >> v2:172.21.15.143:3300/0 conn(0x7f90740a0d60 msgr2=0x7f90740a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:31.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f9073fff700 1 --2- 172.21.15.143:0/2237727534 >> v2:172.21.15.143:3300/0 conn(0x7f90740a0d60 0x7f90740a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:31.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f9073fff700 1 -- 172.21.15.143:0/2237727534 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90740a16c0 con 0x7f9074066ec0 2023-09-29T23:34:31.814 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f9073fff700 1 --2- 172.21.15.143:0/2237727534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9074066ec0 0x7f90740aee60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f90740684c0 tx=0x7f906c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=999241bebbee2748 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:31.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f9072ffd700 1 -- 172.21.15.143:0/2237727534 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c00e070 con 0x7f9074066ec0 2023-09-29T23:34:31.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f9072ffd700 1 -- 172.21.15.143:0/2237727534 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906c00bb40 con 0x7f9074066ec0 2023-09-29T23:34:31.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f9072ffd700 1 -- 172.21.15.143:0/2237727534 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c012430 con 0x7f9074066ec0 2023-09-29T23:34:31.816 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 -- 172.21.15.143:0/2237727534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9074066ec0 msgr2=0x7f90740aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:31.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 --2- 172.21.15.143:0/2237727534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9074066ec0 0x7f90740aee60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f90740684c0 tx=0x7f906c009780 comp rx=0 tx=0).stop 2023-09-29T23:34:31.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 -- 172.21.15.143:0/2237727534 shutdown_connections 2023-09-29T23:34:31.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 --2- 172.21.15.143:0/2237727534 >> v2:172.21.15.143:3300/0 conn(0x7f90740a0d60 0x7f90740a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:31.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 --2- 172.21.15.143:0/2237727534 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 0x7f907413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:31.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 --2- 172.21.15.143:0/2237727534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9074066ec0 0x7f90740aee60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:31.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 -- 172.21.15.143:0/2237727534 >> 172.21.15.143:0/2237727534 conn(0x7f907406f200 msgr2=0x7f9074069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:31.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 -- 172.21.15.143:0/2237727534 shutdown_connections 2023-09-29T23:34:31.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 -- 172.21.15.143:0/2237727534 wait complete. 2023-09-29T23:34:31.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 Processor -- start 2023-09-29T23:34:31.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 -- start start 2023-09-29T23:34:31.817 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9074066ec0 0x7f907406a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90740a0d60 0x7f907414b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 0x7f9074137e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907406e640 con 0x7f9074066ec0 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907406b200 con 0x7f90740a0d60 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.809+0000 7f90794e9700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907406b4a0 con 0x7f907413a300 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f9073fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9074066ec0 0x7f907406a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f9073fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9074066ec0 0x7f907406a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:51126/0 (socket says 172.21.15.143:51126) 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f9073fff700 1 -- 172.21.15.143:0/2380767 learned_addr learned my addr 172.21.15.143:0/2380767 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f9078ce8700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 0x7f9074137e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f90737fe700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90740a0d60 0x7f907414b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f9073fff700 1 -- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 msgr2=0x7f9074137e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:31.818 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f9073fff700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 0x7f9074137e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:31.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f9073fff700 1 -- 172.21.15.143:0/2380767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90740a0d60 msgr2=0x7f907414b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:31.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f9073fff700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90740a0d60 0x7f907414b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:31.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f9073fff700 1 -- 172.21.15.143:0/2380767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f906c009a20 con 0x7f9074066ec0 2023-09-29T23:34:31.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f9078ce8700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 0x7f9074137e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:34:31.819 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f9073fff700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9074066ec0 0x7f907406a9b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f906c009fd0 tx=0x7f906c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:31.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f905bfff700 1 -- 172.21.15.143:0/2380767 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c012cf0 con 0x7f9074066ec0 2023-09-29T23:34:31.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f905bfff700 1 -- 172.21.15.143:0/2380767 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906c012e50 con 0x7f9074066ec0 2023-09-29T23:34:31.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f905bfff700 1 -- 172.21.15.143:0/2380767 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c0276c0 con 0x7f9074066ec0 2023-09-29T23:34:31.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f90794e9700 1 -- 172.21.15.143:0/2380767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907406b760 con 0x7f9074066ec0 2023-09-29T23:34:31.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f90794e9700 1 -- 172.21.15.143:0/2380767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907406d970 con 0x7f9074066ec0 2023-09-29T23:34:31.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.813+0000 7f90794e9700 1 -- 172.21.15.143:0/2380767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907406e780 con 0x7f9074066ec0 2023-09-29T23:34:31.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.817+0000 7f905bfff700 1 -- 172.21.15.143:0/2380767 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f906c024070 con 0x7f9074066ec0 2023-09-29T23:34:31.820 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.817+0000 7f905bfff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.817+0000 7f905bfff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.817+0000 7f905bfff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.817+0000 7f905bfff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.817+0000 7f905bfff700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f906008e8b0 0x7f9060090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:31.821 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.817+0000 7f905bfff700 1 -- 172.21.15.143:0/2380767 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f906c0b6d10 con 0x7f9074066ec0 2023-09-29T23:34:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.817+0000 7f90737fe700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f906008e8b0 0x7f9060090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:31.822 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.817+0000 7f90737fe700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f906008e8b0 0x7f9060090d60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9068006fd0 tx=0x7f9068009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:31.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:31.821+0000 7f905bfff700 1 -- 172.21.15.143:0/2380767 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f906c0876d0 con 0x7f9074066ec0 2023-09-29T23:34:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.013+0000 7f90794e9700 1 -- 172.21.15.143:0/2380767 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f907406e780 con 0x7f9074066ec0 2023-09-29T23:34:32.019 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.013+0000 7f905bfff700 1 -- 172.21.15.143:0/2380767 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f907406e780 con 0x7f9074066ec0 2023-09-29T23:34:32.020 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:32.020 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:32.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 10 mgrc shutdown 2023-09-29T23:34:32.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 -- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f906008e8b0 msgr2=0x7f9060090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:32.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f906008e8b0 0x7f9060090d60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9068006fd0 tx=0x7f9068009250 comp rx=0 tx=0).stop 2023-09-29T23:34:32.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 -- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9074066ec0 msgr2=0x7f907406a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:32.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9074066ec0 0x7f907406a9b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f906c009fd0 tx=0x7f906c019000 comp rx=0 tx=0).stop 2023-09-29T23:34:32.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 -- 172.21.15.143:0/2380767 shutdown_connections 2023-09-29T23:34:32.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f906008e8b0 0x7f9060090d60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:32.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f907413a300 0x7f9074137e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:32.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90740a0d60 0x7f907414b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:32.022 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 --2- 172.21.15.143:0/2380767 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9074066ec0 0x7f907406a9b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:32.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 -- 172.21.15.143:0/2380767 >> 172.21.15.143:0/2380767 conn(0x7f907406f200 msgr2=0x7f907406fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:32.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 -- 172.21.15.143:0/2380767 shutdown_connections 2023-09-29T23:34:32.023 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:32.017+0000 7f90794e9700 1 -- 172.21.15.143:0/2380767 wait complete. 2023-09-29T23:34:35.034 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:35.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.141+0000 7fb346103700 1 Processor -- start 2023-09-29T23:34:35.147 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.141+0000 7fb346103700 1 -- start start 2023-09-29T23:34:35.148 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb346103700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb34013c460 0x7fb34013c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:35.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb346103700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3400b3ae0 con 0x7fb3400b3710 2023-09-29T23:34:35.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb346103700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3400a0090 con 0x7fb34009ebd0 2023-09-29T23:34:35.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb346103700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34009f0e0 con 0x7fb34013c460 2023-09-29T23:34:35.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb345101700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb3400b3710 0x7fb3400a58d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55966/0 (socket says 172.21.15.143:55966) 2023-09-29T23:34:35.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb345101700 1 -- 172.21.15.143:0/2782659209 learned_addr learned my addr 172.21.15.143:0/2782659209 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:34:35.149 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb345902700 1 --2- 172.21.15.143:0/2782659209 >> v2:172.21.15.156:3300/0 conn(0x7fb34013c460 0x7fb34013c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb337fff700 1 -- 172.21.15.143:0/2782659209 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804118262 0 0) 0x7fb3400b3ae0 con 0x7fb3400b3710 2023-09-29T23:34:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb337fff700 1 -- 172.21.15.143:0/2782659209 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb32c003570 con 0x7fb3400b3710 2023-09-29T23:34:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb337fff700 1 -- 172.21.15.143:0/2782659209 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4130651981 0 0) 0x7fb3400a0090 con 0x7fb34009ebd0 2023-09-29T23:34:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb337fff700 1 -- 172.21.15.143:0/2782659209 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3400b3ae0 con 0x7fb34009ebd0 2023-09-29T23:34:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb345902700 1 -- 172.21.15.143:0/2782659209 >> v1:172.21.15.156:6789/0 conn(0x7fb34009ebd0 legacy=0x7fb34009d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb337fff700 1 -- 172.21.15.143:0/2782659209 <== mon.0 v1:172.21.15.143:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 359545147 0 0) 0x7fb32c003570 con 0x7fb3400b3710 2023-09-29T23:34:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb345902700 1 -- 172.21.15.143:0/2782659209 >> v1:172.21.15.143:6789/0 conn(0x7fb3400b3710 legacy=0x7fb3400a58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb345902700 1 -- 172.21.15.143:0/2782659209 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34009f3a0 con 0x7fb34013c460 2023-09-29T23:34:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb345902700 1 --2- 172.21.15.143:0/2782659209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34013c460 0x7fb34013c8e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb338009e60 tx=0x7fb338009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e942e405280839d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:35.151 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb337fff700 1 -- 172.21.15.143:0/2782659209 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800e070 con 0x7fb34013c460 2023-09-29T23:34:35.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb337fff700 1 -- 172.21.15.143:0/2782659209 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33800b7a0 con 0x7fb34013c460 2023-09-29T23:34:35.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb337fff700 1 -- 172.21.15.143:0/2782659209 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c001470 con 0x7fb34013c460 2023-09-29T23:34:35.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb346103700 1 -- 172.21.15.143:0/2782659209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34013c460 msgr2=0x7fb34013c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:35.152 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.145+0000 7fb346103700 1 --2- 172.21.15.143:0/2782659209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34013c460 0x7fb34013c8e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb338009e60 tx=0x7fb338009780 comp rx=0 tx=0).stop 2023-09-29T23:34:35.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 -- 172.21.15.143:0/2782659209 shutdown_connections 2023-09-29T23:34:35.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 --2- 172.21.15.143:0/2782659209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb34013c460 0x7fb34013c8e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:35.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 -- 172.21.15.143:0/2782659209 >> 172.21.15.143:0/2782659209 conn(0x7fb340062ed0 msgr2=0x7fb3400632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:35.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 -- 172.21.15.143:0/2782659209 shutdown_connections 2023-09-29T23:34:35.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 -- 172.21.15.143:0/2782659209 wait complete. 2023-09-29T23:34:35.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 Processor -- start 2023-09-29T23:34:35.153 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 -- start start 2023-09-29T23:34:35.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb34009ebd0 0x7fb3400a2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:35.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3400b3710 0x7fb340136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:35.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb34013c460 0x7fb3401373f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:35.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34006e630 con 0x7fb34013c460 2023-09-29T23:34:35.155 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400a1410 con 0x7fb3400b3710 2023-09-29T23:34:35.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb346103700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3400a16d0 con 0x7fb34009ebd0 2023-09-29T23:34:35.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb345101700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb34009ebd0 0x7fb3400a2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:35.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb345902700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb34013c460 0x7fb3401373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:35.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb345101700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb34009ebd0 0x7fb3400a2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:59638/0 (socket says 172.21.15.143:59638) 2023-09-29T23:34:35.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb345101700 1 -- 172.21.15.143:0/328565039 learned_addr learned my addr 172.21.15.143:0/328565039 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:35.156 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.149+0000 7fb344900700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3400b3710 0x7fb340136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:35.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb345101700 1 -- 172.21.15.143:0/328565039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3400b3710 msgr2=0x7fb340136eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:35.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb345101700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3400b3710 0x7fb340136eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:35.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb345101700 1 -- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb34013c460 msgr2=0x7fb3401373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:35.157 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb345101700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb34013c460 0x7fb3401373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:35.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb345101700 1 -- 172.21.15.143:0/328565039 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb338009a20 con 0x7fb34009ebd0 2023-09-29T23:34:35.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb345902700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb34013c460 0x7fb3401373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:34:35.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb345101700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb34009ebd0 0x7fb3400a2ea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00cba0 tx=0x7fb33c001b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:35.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb3357fa700 1 -- 172.21.15.143:0/328565039 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c007e90 con 0x7fb34009ebd0 2023-09-29T23:34:35.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb3357fa700 1 -- 172.21.15.143:0/328565039 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c007360 con 0x7fb34009ebd0 2023-09-29T23:34:35.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb346103700 1 -- 172.21.15.143:0/328565039 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34013e3a0 con 0x7fb34009ebd0 2023-09-29T23:34:35.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb3357fa700 1 -- 172.21.15.143:0/328565039 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c004830 con 0x7fb34009ebd0 2023-09-29T23:34:35.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb346103700 1 -- 172.21.15.143:0/328565039 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3400a04f0 con 0x7fb34009ebd0 2023-09-29T23:34:35.158 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb346103700 1 -- 172.21.15.143:0/328565039 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34005ecb0 con 0x7fb34009ebd0 2023-09-29T23:34:35.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb3357fa700 1 -- 172.21.15.143:0/328565039 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fb33c004ba0 con 0x7fb34009ebd0 2023-09-29T23:34:35.159 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.153+0000 7fb3357fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:35.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.157+0000 7fb3357fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:35.160 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.157+0000 7fb3357fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:35.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.157+0000 7fb3357fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:35.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.157+0000 7fb3357fa700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb32408e8b0 0x7fb324090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:35.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.157+0000 7fb344900700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb32408e8b0 0x7fb324090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:35.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.157+0000 7fb3357fa700 1 -- 172.21.15.143:0/328565039 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fb33c0b3f80 con 0x7fb34009ebd0 2023-09-29T23:34:35.161 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.157+0000 7fb344900700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb32408e8b0 0x7fb324090d60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb3400637b0 tx=0x7fb330007d50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:35.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.161+0000 7fb3357fa700 1 -- 172.21.15.143:0/328565039 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb33c0848a0 con 0x7fb34009ebd0 2023-09-29T23:34:35.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.361+0000 7fb346103700 1 -- 172.21.15.143:0/328565039 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb34006eb20 con 0x7fb34009ebd0 2023-09-29T23:34:35.366 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.361+0000 7fb3357fa700 1 -- 172.21.15.143:0/328565039 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fb33c018020 con 0x7fb34009ebd0 2023-09-29T23:34:35.366 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:35.366 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 10 mgrc shutdown 2023-09-29T23:34:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 -- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb32408e8b0 msgr2=0x7fb324090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb32408e8b0 0x7fb324090d60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb3400637b0 tx=0x7fb330007d50 comp rx=0 tx=0).stop 2023-09-29T23:34:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 -- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb34009ebd0 msgr2=0x7fb3400a2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:35.368 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb34009ebd0 0x7fb3400a2ea0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00cba0 tx=0x7fb33c001b50 comp rx=0 tx=0).stop 2023-09-29T23:34:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 -- 172.21.15.143:0/328565039 shutdown_connections 2023-09-29T23:34:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb32408e8b0 0x7fb324090d60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb34013c460 0x7fb3401373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3400b3710 0x7fb340136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 --2- 172.21.15.143:0/328565039 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb34009ebd0 0x7fb3400a2ea0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 -- 172.21.15.143:0/328565039 >> 172.21.15.143:0/328565039 conn(0x7fb340062ed0 msgr2=0x7fb34013d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 -- 172.21.15.143:0/328565039 shutdown_connections 2023-09-29T23:34:35.369 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:35.365+0000 7fb346103700 1 -- 172.21.15.143:0/328565039 wait complete. 2023-09-29T23:34:38.379 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:38.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f340dbfb700 1 Processor -- start 2023-09-29T23:34:38.489 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f340dbfb700 1 -- start start 2023-09-29T23:34:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f340dbfb700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f34080b1af0 0x7f34080a7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f340dbfb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3408134610 0x7f34080673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f340dbfb700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34080a7920 con 0x7f34081415d0 2023-09-29T23:34:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f340dbfb700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340806bc10 con 0x7f34080b1af0 2023-09-29T23:34:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f340dbfb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3408141e90 con 0x7f3408134610 2023-09-29T23:34:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f34077fe700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f34080b1af0 0x7f34080a7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f3407fff700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7f34081415d0 0x7f340806cea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:55978/0 (socket says 172.21.15.143:55978) 2023-09-29T23:34:38.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f3407fff700 1 -- 172.21.15.143:0/2760393267 learned_addr learned my addr 172.21.15.143:0/2760393267 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:34:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f3406ffd700 1 --2- 172.21.15.143:0/2760393267 >> v2:172.21.15.156:3300/0 conn(0x7f3408134610 0x7f34080673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f34077fe700 1 -- 172.21.15.143:0/2760393267 >> v2:172.21.15.156:3300/0 conn(0x7f3408134610 msgr2=0x7f34080673a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f34067fc700 1 -- 172.21.15.143:0/2760393267 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782118624 0 0) 0x7f34080a7920 con 0x7f34081415d0 2023-09-29T23:34:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f34077fe700 1 --2- 172.21.15.143:0/2760393267 >> v2:172.21.15.156:3300/0 conn(0x7f3408134610 0x7f34080673a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f34077fe700 1 -- 172.21.15.143:0/2760393267 >> v1:172.21.15.143:6789/0 conn(0x7f34081415d0 legacy=0x7f340806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f34077fe700 1 -- 172.21.15.143:0/2760393267 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3408142150 con 0x7f34080b1af0 2023-09-29T23:34:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f34077fe700 1 --2- 172.21.15.143:0/2760393267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34080b1af0 0x7f34080a7390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f340806eb70 tx=0x7f33fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca961590dc6ecf35 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f34067fc700 1 -- 172.21.15.143:0/2760393267 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc011630 con 0x7f34080b1af0 2023-09-29T23:34:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f34067fc700 1 -- 172.21.15.143:0/2760393267 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc011790 con 0x7f34080b1af0 2023-09-29T23:34:38.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.485+0000 7f34067fc700 1 -- 172.21.15.143:0/2760393267 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc011b30 con 0x7f34080b1af0 2023-09-29T23:34:38.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 -- 172.21.15.143:0/2760393267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34080b1af0 msgr2=0x7f34080a7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:38.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 --2- 172.21.15.143:0/2760393267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34080b1af0 0x7f34080a7390 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f340806eb70 tx=0x7f33fc0091a0 comp rx=0 tx=0).stop 2023-09-29T23:34:38.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 -- 172.21.15.143:0/2760393267 shutdown_connections 2023-09-29T23:34:38.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 --2- 172.21.15.143:0/2760393267 >> v2:172.21.15.156:3300/0 conn(0x7f3408134610 0x7f34080673a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:38.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 --2- 172.21.15.143:0/2760393267 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34080b1af0 0x7f34080a7390 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:38.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 -- 172.21.15.143:0/2760393267 >> 172.21.15.143:0/2760393267 conn(0x7f34080b5420 msgr2=0x7f3408066c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:38.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 -- 172.21.15.143:0/2760393267 shutdown_connections 2023-09-29T23:34:38.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 -- 172.21.15.143:0/2760393267 wait complete. 2023-09-29T23:34:38.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 Processor -- start 2023-09-29T23:34:38.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 -- start start 2023-09-29T23:34:38.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34080b1af0 0x7f34080ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:38.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3408134610 0x7f340814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:38.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34081415d0 0x7f340814e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:38.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340814ea70 con 0x7f34080b1af0 2023-09-29T23:34:38.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34080ac7c0 con 0x7f34081415d0 2023-09-29T23:34:38.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f340dbfb700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34080abbc0 con 0x7f3408134610 2023-09-29T23:34:38.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f34077fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34080b1af0 0x7f34080ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:38.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f34077fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34080b1af0 0x7f34080ac280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:59756/0 (socket says 172.21.15.143:59756) 2023-09-29T23:34:38.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f34077fe700 1 -- 172.21.15.143:0/3451646443 learned_addr learned my addr 172.21.15.143:0/3451646443 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:38.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.489+0000 7f3406ffd700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3408134610 0x7f340814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:38.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f3407fff700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34081415d0 0x7f340814e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:38.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f34077fe700 1 -- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3408134610 msgr2=0x7f340814dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:38.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f34077fe700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3408134610 0x7f340814dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:38.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f34077fe700 1 -- 172.21.15.143:0/3451646443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34081415d0 msgr2=0x7f340814e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:38.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f34077fe700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34081415d0 0x7f340814e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:38.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f34077fe700 1 -- 172.21.15.143:0/3451646443 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33fc009420 con 0x7f34080b1af0 2023-09-29T23:34:38.497 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f34077fe700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34080b1af0 0x7f34080ac280 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009f50 tx=0x7f33fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:38.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f33ef7fe700 1 -- 172.21.15.143:0/3451646443 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc011b10 con 0x7f34080b1af0 2023-09-29T23:34:38.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f33ef7fe700 1 -- 172.21.15.143:0/3451646443 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc0233f0 con 0x7f34080b1af0 2023-09-29T23:34:38.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f340dbfb700 1 -- 172.21.15.143:0/3451646443 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340813f3c0 con 0x7f34080b1af0 2023-09-29T23:34:38.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f33ef7fe700 1 -- 172.21.15.143:0/3451646443 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc023790 con 0x7f34080b1af0 2023-09-29T23:34:38.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f340dbfb700 1 -- 172.21.15.143:0/3451646443 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340814ef10 con 0x7f34080b1af0 2023-09-29T23:34:38.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.493+0000 7f340dbfb700 1 -- 172.21.15.143:0/3451646443 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340805ecb0 con 0x7f34080b1af0 2023-09-29T23:34:38.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.497+0000 7f33ef7fe700 1 -- 172.21.15.143:0/3451646443 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f33fc023b00 con 0x7f34080b1af0 2023-09-29T23:34:38.499 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.497+0000 7f33ef7fe700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:38.500 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.497+0000 7f33ef7fe700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:38.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.497+0000 7f33ef7fe700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:38.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.497+0000 7f33ef7fe700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:38.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.497+0000 7f33ef7fe700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f33f408e8b0 0x7f33f4090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:38.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.497+0000 7f33ef7fe700 1 -- 172.21.15.143:0/3451646443 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f33fc0b5a90 con 0x7f34080b1af0 2023-09-29T23:34:38.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.497+0000 7f3406ffd700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f33f408e8b0 0x7f33f4090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:38.501 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.497+0000 7f3406ffd700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f33f408e8b0 0x7f33f4090d60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009fd0 tx=0x7f33f800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:38.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.501+0000 7f33ef7fe700 1 -- 172.21.15.143:0/3451646443 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f33fc059be0 con 0x7f34080b1af0 2023-09-29T23:34:38.698 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.693+0000 7f340dbfb700 1 -- 172.21.15.143:0/3451646443 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3408142410 con 0x7f34080b1af0 2023-09-29T23:34:38.699 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.693+0000 7f33ef7fe700 1 -- 172.21.15.143:0/3451646443 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f33fc015080 con 0x7f34080b1af0 2023-09-29T23:34:38.699 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:38.699 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:38.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 10 mgrc shutdown 2023-09-29T23:34:38.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 -- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f33f408e8b0 msgr2=0x7f33f4090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:38.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f33f408e8b0 0x7f33f4090d60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009fd0 tx=0x7f33f800b040 comp rx=0 tx=0).stop 2023-09-29T23:34:38.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 -- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34080b1af0 msgr2=0x7f34080ac280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:38.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34080b1af0 0x7f34080ac280 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f33fc009f50 tx=0x7f33fc017000 comp rx=0 tx=0).stop 2023-09-29T23:34:38.702 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 -- 172.21.15.143:0/3451646443 shutdown_connections 2023-09-29T23:34:38.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f33f408e8b0 0x7f33f4090d60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:38.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34081415d0 0x7f340814e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:38.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f3408134610 0x7f340814dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:38.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 --2- 172.21.15.143:0/3451646443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f34080b1af0 0x7f34080ac280 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:38.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 -- 172.21.15.143:0/3451646443 >> 172.21.15.143:0/3451646443 conn(0x7f34080b5420 msgr2=0x7f34080a4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:38.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 -- 172.21.15.143:0/3451646443 shutdown_connections 2023-09-29T23:34:38.703 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:38.697+0000 7f340dbfb700 1 -- 172.21.15.143:0/3451646443 wait complete. 2023-09-29T23:34:41.713 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:41.825 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f734eccb700 1 Processor -- start 2023-09-29T23:34:41.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f734eccb700 1 -- start start 2023-09-29T23:34:41.826 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f734eccb700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 0x7f73481408f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:41.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f734eccb700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7348132cb0 0x7f73481330a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:41.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f734eccb700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7348140e30 con 0x7f73480a57f0 2023-09-29T23:34:41.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f734eccb700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734806d710 con 0x7f7348132cb0 2023-09-29T23:34:41.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f734eccb700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73480a5bc0 con 0x7f7348140500 2023-09-29T23:34:41.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f7347fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 0x7f73481408f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:41.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f7347fff700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 0x7f73481408f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:59662/0 (socket says 172.21.15.143:59662) 2023-09-29T23:34:41.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f734ca67700 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f73480a57f0 0x7f73480a6760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.143:39220/0 (socket says 172.21.15.143:39220) 2023-09-29T23:34:41.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f7347fff700 1 -- 172.21.15.143:0/2777040985 learned_addr learned my addr 172.21.15.143:0/2777040985 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:41.827 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f734d268700 1 --2- 172.21.15.143:0/2777040985 >> v2:172.21.15.156:3300/0 conn(0x7f7348132cb0 0x7f73481330a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f73477fe700 1 -- 172.21.15.143:0/2777040985 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69383401 0 0) 0x7f7348140e30 con 0x7f73480a57f0 2023-09-29T23:34:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f7347fff700 1 -- 172.21.15.143:0/2777040985 >> v2:172.21.15.156:3300/0 conn(0x7f7348132cb0 msgr2=0x7f73481330a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f7347fff700 1 --2- 172.21.15.143:0/2777040985 >> v2:172.21.15.156:3300/0 conn(0x7f7348132cb0 0x7f73481330a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f7347fff700 1 -- 172.21.15.143:0/2777040985 >> v1:172.21.15.156:6789/0 conn(0x7f73480a57f0 legacy=0x7f73480a6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.825+0000 7f7347fff700 1 -- 172.21.15.143:0/2777040985 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7348141080 con 0x7f7348140500 2023-09-29T23:34:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f7347fff700 1 --2- 172.21.15.143:0/2777040985 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 0x7f73481408f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f733c0099b0 tx=0x7f733c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f085b487720b4e85 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f73477fe700 1 -- 172.21.15.143:0/2777040985 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c009ea0 con 0x7f7348140500 2023-09-29T23:34:41.829 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f73477fe700 1 -- 172.21.15.143:0/2777040985 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733c00b860 con 0x7f7348140500 2023-09-29T23:34:41.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f73477fe700 1 -- 172.21.15.143:0/2777040985 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733c00bc00 con 0x7f7348140500 2023-09-29T23:34:41.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 -- 172.21.15.143:0/2777040985 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 msgr2=0x7f73481408f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:41.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 --2- 172.21.15.143:0/2777040985 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 0x7f73481408f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f733c0099b0 tx=0x7f733c009250 comp rx=0 tx=0).stop 2023-09-29T23:34:41.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 -- 172.21.15.143:0/2777040985 shutdown_connections 2023-09-29T23:34:41.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 --2- 172.21.15.143:0/2777040985 >> v2:172.21.15.156:3300/0 conn(0x7f7348132cb0 0x7f73481330a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:41.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 --2- 172.21.15.143:0/2777040985 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 0x7f73481408f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:41.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 -- 172.21.15.143:0/2777040985 >> 172.21.15.143:0/2777040985 conn(0x7f7348067dc0 msgr2=0x7f73480b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:41.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 -- 172.21.15.143:0/2777040985 shutdown_connections 2023-09-29T23:34:41.830 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 -- 172.21.15.143:0/2777040985 wait complete. 2023-09-29T23:34:41.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 Processor -- start 2023-09-29T23:34:41.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 -- start start 2023-09-29T23:34:41.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73480a57f0 0x7f73480acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:41.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7348132cb0 0x7f73480ae300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:41.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 0x7f7348145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:41.832 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480ad9a0 con 0x7f73480a57f0 2023-09-29T23:34:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480adba0 con 0x7f7348132cb0 2023-09-29T23:34:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734eccb700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73481535d0 con 0x7f7348140500 2023-09-29T23:34:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734d268700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 0x7f7348145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734ca67700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73480a57f0 0x7f73480acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734d268700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 0x7f7348145ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:59670/0 (socket says 172.21.15.143:59670) 2023-09-29T23:34:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734d268700 1 -- 172.21.15.143:0/4049418872 learned_addr learned my addr 172.21.15.143:0/4049418872 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f7347fff700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7348132cb0 0x7f73480ae300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734ca67700 1 -- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 msgr2=0x7f7348145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734ca67700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 0x7f7348145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734ca67700 1 -- 172.21.15.143:0/4049418872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7348132cb0 msgr2=0x7f73480ae300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:41.833 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734ca67700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7348132cb0 0x7f73480ae300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.829+0000 7f734ca67700 1 -- 172.21.15.143:0/4049418872 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c0094f0 con 0x7f73480a57f0 2023-09-29T23:34:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.833+0000 7f734ca67700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73480a57f0 0x7f73480acee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f734000cd50 tx=0x7f7340001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.833+0000 7f7344ff9700 1 -- 172.21.15.143:0/4049418872 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340005df0 con 0x7f73480a57f0 2023-09-29T23:34:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.833+0000 7f7344ff9700 1 -- 172.21.15.143:0/4049418872 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734000a870 con 0x7f73480a57f0 2023-09-29T23:34:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.833+0000 7f734eccb700 1 -- 172.21.15.143:0/4049418872 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7348153890 con 0x7f73480a57f0 2023-09-29T23:34:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.833+0000 7f7344ff9700 1 -- 172.21.15.143:0/4049418872 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734000ac30 con 0x7f73480a57f0 2023-09-29T23:34:41.834 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.833+0000 7f734eccb700 1 -- 172.21.15.143:0/4049418872 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73480b2460 con 0x7f73480a57f0 2023-09-29T23:34:41.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.833+0000 7f734eccb700 1 -- 172.21.15.143:0/4049418872 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73480b1950 con 0x7f73480a57f0 2023-09-29T23:34:41.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.833+0000 7f7344ff9700 1 -- 172.21.15.143:0/4049418872 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f73400070f0 con 0x7f73480a57f0 2023-09-29T23:34:41.835 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.833+0000 7f7344ff9700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:41.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.837+0000 7f7344ff9700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:41.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.837+0000 7f7344ff9700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:41.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.837+0000 7f7344ff9700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:41.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.837+0000 7f7344ff9700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f733408e8e0 0x7f7334090d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:41.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.837+0000 7f7347fff700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f733408e8e0 0x7f7334090d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:41.836 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.837+0000 7f7344ff9700 1 -- 172.21.15.143:0/4049418872 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f73400b4920 con 0x7f73480a57f0 2023-09-29T23:34:41.837 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.837+0000 7f7347fff700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f733408e8e0 0x7f7334090d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f733c0099b0 tx=0x7f733c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:41.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:41.841+0000 7f7344ff9700 1 -- 172.21.15.143:0/4049418872 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f73400852c0 con 0x7f73480a57f0 2023-09-29T23:34:42.036 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 1 -- 172.21.15.143:0/4049418872 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f734805ecb0 con 0x7f73480a57f0 2023-09-29T23:34:42.037 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f7344ff9700 1 -- 172.21.15.143:0/4049418872 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f73400888e0 con 0x7f73480a57f0 2023-09-29T23:34:42.037 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:42.037 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:42.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 10 mgrc shutdown 2023-09-29T23:34:42.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 1 -- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f733408e8e0 msgr2=0x7f7334090d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:42.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f733408e8e0 0x7f7334090d90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f733c0099b0 tx=0x7f733c017000 comp rx=0 tx=0).stop 2023-09-29T23:34:42.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 1 -- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73480a57f0 msgr2=0x7f73480acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:42.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73480a57f0 0x7f73480acee0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f734000cd50 tx=0x7f7340001b00 comp rx=0 tx=0).stop 2023-09-29T23:34:42.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 1 -- 172.21.15.143:0/4049418872 shutdown_connections 2023-09-29T23:34:42.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f733408e8e0 0x7f7334090d90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:42.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f7348140500 0x7f7348145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:42.040 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7348132cb0 0x7f73480ae300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:42.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 1 --2- 172.21.15.143:0/4049418872 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f73480a57f0 0x7f73480acee0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:42.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.037+0000 7f734eccb700 1 -- 172.21.15.143:0/4049418872 >> 172.21.15.143:0/4049418872 conn(0x7f7348067dc0 msgr2=0x7f734806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:42.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.041+0000 7f734eccb700 1 -- 172.21.15.143:0/4049418872 shutdown_connections 2023-09-29T23:34:42.041 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:42.041+0000 7f734eccb700 1 -- 172.21.15.143:0/4049418872 wait complete. 2023-09-29T23:34:45.052 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:45.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdcd6df700 1 Processor -- start 2023-09-29T23:34:45.165 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdcd6df700 1 -- start start 2023-09-29T23:34:45.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdcd6df700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efdc8066ec0 0x7efdc80a1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:45.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdcd6df700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efdc809f830 0x7efdc813d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:45.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdcd6df700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc80a16b0 con 0x7efdc809fc70 2023-09-29T23:34:45.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdcd6df700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc8067290 con 0x7efdc8066ec0 2023-09-29T23:34:45.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdcd6df700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc80a0510 con 0x7efdc809f830 2023-09-29T23:34:45.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc7fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efdc8066ec0 0x7efdc80a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:45.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdccede700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7efdc809fc70 0x7efdc8141220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34520/0 (socket says 172.21.15.143:34520) 2023-09-29T23:34:45.167 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc7fff700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7efdc8066ec0 0x7efdc80a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43430/0 (socket says 172.21.15.143:43430) 2023-09-29T23:34:45.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdccede700 1 -- 172.21.15.143:0/2429914451 learned_addr learned my addr 172.21.15.143:0/2429914451 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:34:45.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc77fe700 1 --2- 172.21.15.143:0/2429914451 >> v2:172.21.15.156:3300/0 conn(0x7efdc809f830 0x7efdc813d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:45.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc7fff700 1 -- 172.21.15.143:0/2429914451 >> v2:172.21.15.156:3300/0 conn(0x7efdc809f830 msgr2=0x7efdc813d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:45.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc7fff700 1 --2- 172.21.15.143:0/2429914451 >> v2:172.21.15.156:3300/0 conn(0x7efdc809f830 0x7efdc813d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:45.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc7fff700 1 -- 172.21.15.143:0/2429914451 >> v1:172.21.15.143:6789/0 conn(0x7efdc809fc70 legacy=0x7efdc8141220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:45.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc6ffd700 1 -- 172.21.15.143:0/2429914451 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051398350 0 0) 0x7efdc80a16b0 con 0x7efdc809fc70 2023-09-29T23:34:45.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc7fff700 1 -- 172.21.15.143:0/2429914451 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc806e400 con 0x7efdc8066ec0 2023-09-29T23:34:45.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc7fff700 1 --2- 172.21.15.143:0/2429914451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc8066ec0 0x7efdc80a1170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efdc813ede0 tx=0x7efdbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8085bc88162466b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:45.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc6ffd700 1 -- 172.21.15.143:0/2429914451 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc011630 con 0x7efdc8066ec0 2023-09-29T23:34:45.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc6ffd700 1 -- 172.21.15.143:0/2429914451 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc011790 con 0x7efdc8066ec0 2023-09-29T23:34:45.168 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.165+0000 7efdc6ffd700 1 -- 172.21.15.143:0/2429914451 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc011b30 con 0x7efdc8066ec0 2023-09-29T23:34:45.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 -- 172.21.15.143:0/2429914451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc8066ec0 msgr2=0x7efdc80a1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:45.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 --2- 172.21.15.143:0/2429914451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc8066ec0 0x7efdc80a1170 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efdc813ede0 tx=0x7efdbc0091a0 comp rx=0 tx=0).stop 2023-09-29T23:34:45.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 -- 172.21.15.143:0/2429914451 shutdown_connections 2023-09-29T23:34:45.171 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 --2- 172.21.15.143:0/2429914451 >> v2:172.21.15.156:3300/0 conn(0x7efdc809f830 0x7efdc813d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:45.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 --2- 172.21.15.143:0/2429914451 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc8066ec0 0x7efdc80a1170 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:45.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 -- 172.21.15.143:0/2429914451 >> 172.21.15.143:0/2429914451 conn(0x7efdc806f200 msgr2=0x7efdc8069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:45.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 -- 172.21.15.143:0/2429914451 shutdown_connections 2023-09-29T23:34:45.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 -- 172.21.15.143:0/2429914451 wait complete. 2023-09-29T23:34:45.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 Processor -- start 2023-09-29T23:34:45.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 -- start start 2023-09-29T23:34:45.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc8066ec0 0x7efdc806b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:45.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efdc809f830 0x7efdc806c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:45.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdc809fc70 0x7efdc8139ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:45.172 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc8139550 con 0x7efdc8066ec0 2023-09-29T23:34:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc8134fc0 con 0x7efdc809fc70 2023-09-29T23:34:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc809ebd0 con 0x7efdc809f830 2023-09-29T23:34:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdc77fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efdc809f830 0x7efdc806c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdc77fe700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efdc809f830 0x7efdc806c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:42548/0 (socket says 172.21.15.143:42548) 2023-09-29T23:34:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdc7fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc8066ec0 0x7efdc806b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdc77fe700 1 -- 172.21.15.143:0/1356782306 learned_addr learned my addr 172.21.15.143:0/1356782306 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdccede700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdc809fc70 0x7efdc8139ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdc7fff700 1 -- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efdc809f830 msgr2=0x7efdc806c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdc7fff700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efdc809f830 0x7efdc806c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:45.173 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdc7fff700 1 -- 172.21.15.143:0/1356782306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdc809fc70 msgr2=0x7efdc8139ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdc7fff700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdc809fc70 0x7efdc8139ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdc7fff700 1 -- 172.21.15.143:0/1356782306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdbc009420 con 0x7efdc8066ec0 2023-09-29T23:34:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdc77fe700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efdc809f830 0x7efdc806c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:34:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdc7fff700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc8066ec0 0x7efdc806b770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efdbc016000 tx=0x7efdbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdaffff700 1 -- 172.21.15.143:0/1356782306 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc000d10 con 0x7efdc8066ec0 2023-09-29T23:34:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 -- 172.21.15.143:0/1356782306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc806ca10 con 0x7efdc8066ec0 2023-09-29T23:34:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdaffff700 1 -- 172.21.15.143:0/1356782306 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdbc011b10 con 0x7efdc8066ec0 2023-09-29T23:34:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdaffff700 1 -- 172.21.15.143:0/1356782306 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdbc025640 con 0x7efdc8066ec0 2023-09-29T23:34:45.174 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.169+0000 7efdcd6df700 1 -- 172.21.15.143:0/1356782306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc81391d0 con 0x7efdc8066ec0 2023-09-29T23:34:45.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.173+0000 7efdcd6df700 1 -- 172.21.15.143:0/1356782306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc8135100 con 0x7efdc8066ec0 2023-09-29T23:34:45.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.177+0000 7efdaffff700 1 -- 172.21.15.143:0/1356782306 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7efdbc0193b0 con 0x7efdc8066ec0 2023-09-29T23:34:45.175 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.177+0000 7efdaffff700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:45.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.177+0000 7efdaffff700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:45.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.177+0000 7efdaffff700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:45.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.177+0000 7efdaffff700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:45.176 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.177+0000 7efdaffff700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efdb408e830 0x7efdb4090ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:45.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.177+0000 7efdc77fe700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efdb408e830 0x7efdb4090ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:45.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.177+0000 7efdaffff700 1 -- 172.21.15.143:0/1356782306 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7efdbc0b6440 con 0x7efdc8066ec0 2023-09-29T23:34:45.177 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.177+0000 7efdc77fe700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efdb408e830 0x7efdb4090ce0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efdb80099f0 tx=0x7efdb800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:45.181 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.181+0000 7efdaffff700 1 -- 172.21.15.143:0/1356782306 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7efdbc086e00 con 0x7efdc8066ec0 2023-09-29T23:34:45.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.373+0000 7efdcd6df700 1 -- 172.21.15.143:0/1356782306 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7efdc805ecb0 con 0x7efdc8066ec0 2023-09-29T23:34:45.375 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdaffff700 1 -- 172.21.15.143:0/1356782306 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7efdbc015100 con 0x7efdc8066ec0 2023-09-29T23:34:45.376 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:45.376 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:45.378 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 10 mgrc shutdown 2023-09-29T23:34:45.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 -- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efdb408e830 msgr2=0x7efdb4090ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:45.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efdb408e830 0x7efdb4090ce0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7efdb80099f0 tx=0x7efdb800b040 comp rx=0 tx=0).stop 2023-09-29T23:34:45.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 -- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc8066ec0 msgr2=0x7efdc806b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:45.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc8066ec0 0x7efdc806b770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efdbc016000 tx=0x7efdbc017000 comp rx=0 tx=0).stop 2023-09-29T23:34:45.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 -- 172.21.15.143:0/1356782306 shutdown_connections 2023-09-29T23:34:45.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7efdb408e830 0x7efdb4090ce0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:45.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efdc809fc70 0x7efdc8139ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:45.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7efdc809f830 0x7efdc806c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:45.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 --2- 172.21.15.143:0/1356782306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efdc8066ec0 0x7efdc806b770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:45.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 -- 172.21.15.143:0/1356782306 >> 172.21.15.143:0/1356782306 conn(0x7efdc806f200 msgr2=0x7efdc80a0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:45.379 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 -- 172.21.15.143:0/1356782306 shutdown_connections 2023-09-29T23:34:45.380 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:45.377+0000 7efdcd6df700 1 -- 172.21.15.143:0/1356782306 wait complete. 2023-09-29T23:34:48.390 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4ec005700 1 Processor -- start 2023-09-29T23:34:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4ec005700 1 -- start start 2023-09-29T23:34:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4ec005700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fb4e40a57f0 0x7fb4e40a6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4ec005700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb4e41333c0 0x7fb4e4140920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4ec005700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e40a5bc0 con 0x7fb4e4140500 2023-09-29T23:34:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4ec005700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e41337f0 con 0x7fb4e40a57f0 2023-09-29T23:34:48.506 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4ec005700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e406d710 con 0x7fb4e41333c0 2023-09-29T23:34:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4e95a0700 1 --1- >> v1:172.21.15.143:6789/0 conn(0x7fb4e4140500 0x7fb4e4132cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.143:6789/0 says I am v1:172.21.15.143:34534/0 (socket says 172.21.15.143:34534) 2023-09-29T23:34:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4e95a0700 1 -- 172.21.15.143:0/3445388046 learned_addr learned my addr 172.21.15.143:0/3445388046 (peer_addr_for_me v1:172.21.15.143:0/0) 2023-09-29T23:34:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4e9da1700 1 --2- 172.21.15.143:0/3445388046 >> v2:172.21.15.143:3300/0 conn(0x7fb4e40a57f0 0x7fb4e40a6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:48.507 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4ea5a2700 1 --2- 172.21.15.143:0/3445388046 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb4e41333c0 0x7fb4e4140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4e8d9f700 1 -- 172.21.15.143:0/3445388046 <== mon.0 v1:172.21.15.143:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 566458828 0 0) 0x7fb4e40a5bc0 con 0x7fb4e4140500 2023-09-29T23:34:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4e8d9f700 1 -- 172.21.15.143:0/3445388046 --> v1:172.21.15.143:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4d0003570 con 0x7fb4e4140500 2023-09-29T23:34:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4e9da1700 1 -- 172.21.15.143:0/3445388046 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb4e41333c0 msgr2=0x7fb4e4140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4e9da1700 1 --2- 172.21.15.143:0/3445388046 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb4e41333c0 0x7fb4e4140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4e9da1700 1 -- 172.21.15.143:0/3445388046 >> v1:172.21.15.143:6789/0 conn(0x7fb4e4140500 legacy=0x7fb4e4132cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:48.508 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.505+0000 7fb4e9da1700 1 -- 172.21.15.143:0/3445388046 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e4141080 con 0x7fb4e40a57f0 2023-09-29T23:34:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4e9da1700 1 --2- 172.21.15.143:0/3445388046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4e40a57f0 0x7fb4e40a6760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009910 tx=0x7fb4e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2088bddb60ae96f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4e8d9f700 1 -- 172.21.15.143:0/3445388046 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0011630 con 0x7fb4e40a57f0 2023-09-29T23:34:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4e8d9f700 1 -- 172.21.15.143:0/3445388046 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e0011790 con 0x7fb4e40a57f0 2023-09-29T23:34:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4e8d9f700 1 -- 172.21.15.143:0/3445388046 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0011b30 con 0x7fb4e40a57f0 2023-09-29T23:34:48.509 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4ec005700 1 -- 172.21.15.143:0/3445388046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4e40a57f0 msgr2=0x7fb4e40a6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4ec005700 1 --2- 172.21.15.143:0/3445388046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4e40a57f0 0x7fb4e40a6760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009910 tx=0x7fb4e00091a0 comp rx=0 tx=0).stop 2023-09-29T23:34:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4ec005700 1 -- 172.21.15.143:0/3445388046 shutdown_connections 2023-09-29T23:34:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4ec005700 1 --2- 172.21.15.143:0/3445388046 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb4e41333c0 0x7fb4e4140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4ec005700 1 --2- 172.21.15.143:0/3445388046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4e40a57f0 0x7fb4e40a6760 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4ec005700 1 -- 172.21.15.143:0/3445388046 >> 172.21.15.143:0/3445388046 conn(0x7fb4e4067dc0 msgr2=0x7fb4e40b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4ec005700 1 -- 172.21.15.143:0/3445388046 shutdown_connections 2023-09-29T23:34:48.510 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4ec005700 1 -- 172.21.15.143:0/3445388046 wait complete. 2023-09-29T23:34:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4ec005700 1 Processor -- start 2023-09-29T23:34:48.511 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.509+0000 7fb4ec005700 1 -- start start 2023-09-29T23:34:48.513 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ec005700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb4e41333c0 0x7fb4e40a44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ec005700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4e4140500 0x7fb4e40aca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ec005700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4e412f540 0x7fb4e412f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ec005700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40ac190 con 0x7fb4e412f540 2023-09-29T23:34:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ec005700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e4151180 con 0x7fb4e4140500 2023-09-29T23:34:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ec005700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e4151420 con 0x7fb4e41333c0 2023-09-29T23:34:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4e9da1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb4e41333c0 0x7fb4e40a44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ea5a2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4e412f540 0x7fb4e412f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ea5a2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4e412f540 0x7fb4e412f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43466/0 (socket says 172.21.15.143:43466) 2023-09-29T23:34:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ea5a2700 1 -- 172.21.15.143:0/538301418 learned_addr learned my addr 172.21.15.143:0/538301418 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:48.514 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4e95a0700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4e4140500 0x7fb4e40aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ea5a2700 1 -- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb4e41333c0 msgr2=0x7fb4e40a44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ea5a2700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb4e41333c0 0x7fb4e40a44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ea5a2700 1 -- 172.21.15.143:0/538301418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4e4140500 msgr2=0x7fb4e40aca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ea5a2700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4e4140500 0x7fb4e40aca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ea5a2700 1 -- 172.21.15.143:0/538301418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e0009420 con 0x7fb4e412f540 2023-09-29T23:34:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4e95a0700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4e4140500 0x7fb4e40aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:34:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ea5a2700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4e412f540 0x7fb4e412f990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009e90 tx=0x7fb4dc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4d9ffb700 1 -- 172.21.15.143:0/538301418 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00cc20 con 0x7fb4e412f540 2023-09-29T23:34:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4d9ffb700 1 -- 172.21.15.143:0/538301418 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc00f040 con 0x7fb4e412f540 2023-09-29T23:34:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ec005700 1 -- 172.21.15.143:0/538301418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e41516e0 con 0x7fb4e412f540 2023-09-29T23:34:48.515 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4d9ffb700 1 -- 172.21.15.143:0/538301418 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc014480 con 0x7fb4e412f540 2023-09-29T23:34:48.516 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.513+0000 7fb4ec005700 1 -- 172.21.15.143:0/538301418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e40ab7f0 con 0x7fb4e412f540 2023-09-29T23:34:48.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.517+0000 7fb4d9ffb700 1 -- 172.21.15.143:0/538301418 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fb4dc014830 con 0x7fb4e412f540 2023-09-29T23:34:48.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.517+0000 7fb4d9ffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:48.517 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.517+0000 7fb4ec005700 1 -- 172.21.15.143:0/538301418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e405ecb0 con 0x7fb4e412f540 2023-09-29T23:34:48.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.517+0000 7fb4d9ffb700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:48.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.517+0000 7fb4d9ffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:48.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.517+0000 7fb4d9ffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:48.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.517+0000 7fb4d9ffb700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4c808e8b0 0x7fb4c8090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:48.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.517+0000 7fb4d9ffb700 1 -- 172.21.15.143:0/538301418 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fb4dc0b52f0 con 0x7fb4e412f540 2023-09-29T23:34:48.518 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.517+0000 7fb4e9da1700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4c808e8b0 0x7fb4c8090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:48.519 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.517+0000 7fb4e9da1700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4c808e8b0 0x7fb4c8090d60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb4e406e530 tx=0x7fb4e0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:48.525 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.525+0000 7fb4d9ffb700 1 -- 172.21.15.143:0/538301418 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fb4dc089740 con 0x7fb4e412f540 2023-09-29T23:34:48.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.713+0000 7fb4ec005700 1 -- 172.21.15.143:0/538301418 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb4e404b5c0 con 0x7fb4e412f540 2023-09-29T23:34:48.714 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.713+0000 7fb4d9ffb700 1 -- 172.21.15.143:0/538301418 <== mon.0 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7fb4dc018100 con 0x7fb4e412f540 2023-09-29T23:34:48.715 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:48.715 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:48.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 10 mgrc shutdown 2023-09-29T23:34:48.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 -- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4c808e8b0 msgr2=0x7fb4c8090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:48.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4c808e8b0 0x7fb4c8090d60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb4e406e530 tx=0x7fb4e0017000 comp rx=0 tx=0).stop 2023-09-29T23:34:48.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 -- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4e412f540 msgr2=0x7fb4e412f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:48.717 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4e412f540 0x7fb4e412f990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009e90 tx=0x7fb4dc00b040 comp rx=0 tx=0).stop 2023-09-29T23:34:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 -- 172.21.15.143:0/538301418 shutdown_connections 2023-09-29T23:34:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fb4c808e8b0 0x7fb4c8090d60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb4e412f540 0x7fb4e412f990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4e4140500 0x7fb4e40aca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 --2- 172.21.15.143:0/538301418 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fb4e41333c0 0x7fb4e40a44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 -- 172.21.15.143:0/538301418 >> 172.21.15.143:0/538301418 conn(0x7fb4e4067dc0 msgr2=0x7fb4e406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 -- 172.21.15.143:0/538301418 shutdown_connections 2023-09-29T23:34:48.718 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:48.717+0000 7fb4ec005700 1 -- 172.21.15.143:0/538301418 wait complete. 2023-09-29T23:34:51.730 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-29T23:34:51.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.841+0000 7f1362526700 1 Processor -- start 2023-09-29T23:34:51.842 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.841+0000 7f1362526700 1 -- start start 2023-09-29T23:34:51.843 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.841+0000 7f1362526700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06ee30 0x7f135c09cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:51.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.841+0000 7f1362526700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7f135c13fbb0 0x7f135c140300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:51.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 -- --> v1:172.21.15.143:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f135c143460 con 0x7f135c06f200 2023-09-29T23:34:51.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f135c09bca0 con 0x7f135c13fbb0 2023-09-29T23:34:51.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135c09d170 con 0x7f135c06ee30 2023-09-29T23:34:51.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1361524700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06ee30 0x7f135c09cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:51.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1361524700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06ee30 0x7f135c09cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:42560/0 (socket says 172.21.15.143:42560) 2023-09-29T23:34:51.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1361524700 1 -- 172.21.15.143:0/1116771988 learned_addr learned my addr 172.21.15.143:0/1116771988 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:51.844 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1361d25700 1 --2- 172.21.15.143:0/1116771988 >> v2:172.21.15.143:3300/0 conn(0x7f135c13fbb0 0x7f135c140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:51.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1361524700 1 -- 172.21.15.143:0/1116771988 >> v2:172.21.15.143:3300/0 conn(0x7f135c13fbb0 msgr2=0x7f135c140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:51.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1361524700 1 --2- 172.21.15.143:0/1116771988 >> v2:172.21.15.143:3300/0 conn(0x7f135c13fbb0 0x7f135c140300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:51.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1361524700 1 -- 172.21.15.143:0/1116771988 >> v1:172.21.15.143:6789/0 conn(0x7f135c06f200 legacy=0x7f135c13f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:51.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1361524700 1 -- 172.21.15.143:0/1116771988 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c09d2e0 con 0x7f135c06ee30 2023-09-29T23:34:51.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1361524700 1 --2- 172.21.15.143:0/1116771988 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06ee30 0x7f135c09cc30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f13580099b0 tx=0x7f1358009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31d8afe5caa640ea server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:51.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1353fff700 1 -- 172.21.15.143:0/1116771988 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358009ea0 con 0x7f135c06ee30 2023-09-29T23:34:51.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1353fff700 1 -- 172.21.15.143:0/1116771988 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135800b860 con 0x7f135c06ee30 2023-09-29T23:34:51.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1353fff700 1 -- 172.21.15.143:0/1116771988 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135800bc00 con 0x7f135c06ee30 2023-09-29T23:34:51.846 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 -- 172.21.15.143:0/1116771988 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06ee30 msgr2=0x7f135c09cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:51.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 --2- 172.21.15.143:0/1116771988 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06ee30 0x7f135c09cc30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f13580099b0 tx=0x7f1358009250 comp rx=0 tx=0).stop 2023-09-29T23:34:51.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 -- 172.21.15.143:0/1116771988 shutdown_connections 2023-09-29T23:34:51.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 --2- 172.21.15.143:0/1116771988 >> v2:172.21.15.143:3300/0 conn(0x7f135c13fbb0 0x7f135c140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:51.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 --2- 172.21.15.143:0/1116771988 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06ee30 0x7f135c09cc30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:51.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 -- 172.21.15.143:0/1116771988 >> 172.21.15.143:0/1116771988 conn(0x7f135c0b2ca0 msgr2=0x7f135c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:51.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 -- 172.21.15.143:0/1116771988 shutdown_connections 2023-09-29T23:34:51.847 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 -- 172.21.15.143:0/1116771988 wait complete. 2023-09-29T23:34:51.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 Processor -- start 2023-09-29T23:34:51.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.845+0000 7f1362526700 1 -- start start 2023-09-29T23:34:51.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1362526700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f135c06ee30 0x7f135c0a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:51.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1362526700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06f200 0x7f135c0a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:51.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1362526700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f135c13fbb0 0x7f135c0a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:51.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1362526700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c0a7970 con 0x7f135c06ee30 2023-09-29T23:34:51.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1362526700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f135c0a7330 con 0x7f135c13fbb0 2023-09-29T23:34:51.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1362526700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135c1536c0 con 0x7f135c06f200 2023-09-29T23:34:51.849 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1361524700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f135c06ee30 0x7f135c0a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:51.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1360d23700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06f200 0x7f135c0a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:51.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1361524700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f135c06ee30 0x7f135c0a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43476/0 (socket says 172.21.15.143:43476) 2023-09-29T23:34:51.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1360d23700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06f200 0x7f135c0a8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:42562/0 (socket says 172.21.15.143:42562) 2023-09-29T23:34:51.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1361524700 1 -- 172.21.15.143:0/1072602023 learned_addr learned my addr 172.21.15.143:0/1072602023 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:51.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1361d25700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f135c13fbb0 0x7f135c0a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:51.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1360d23700 1 -- 172.21.15.143:0/1072602023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f135c13fbb0 msgr2=0x7f135c0a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:51.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1360d23700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f135c13fbb0 0x7f135c0a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:51.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1360d23700 1 -- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f135c06ee30 msgr2=0x7f135c0a6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:51.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1360d23700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f135c06ee30 0x7f135c0a6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:51.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1360d23700 1 -- 172.21.15.143:0/1072602023 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13580094f0 con 0x7f135c06f200 2023-09-29T23:34:51.850 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1361d25700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f135c13fbb0 0x7f135c0a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:34:51.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1360d23700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06f200 0x7f135c0a8070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c5c0 tx=0x7f134c007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:51.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f13517fa700 1 -- 172.21.15.143:0/1072602023 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c011590 con 0x7f135c06f200 2023-09-29T23:34:51.851 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f13517fa700 1 -- 172.21.15.143:0/1072602023 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c011710 con 0x7f135c06f200 2023-09-29T23:34:51.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1362526700 1 -- 172.21.15.143:0/1072602023 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f135c1538f0 con 0x7f135c06f200 2023-09-29T23:34:51.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f13517fa700 1 -- 172.21.15.143:0/1072602023 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c011ad0 con 0x7f135c06f200 2023-09-29T23:34:51.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1362526700 1 -- 172.21.15.143:0/1072602023 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f135c1511b0 con 0x7f135c06f200 2023-09-29T23:34:51.852 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.849+0000 7f1362526700 1 -- 172.21.15.143:0/1072602023 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f135c05ecb0 con 0x7f135c06f200 2023-09-29T23:34:51.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.853+0000 7f13517fa700 1 -- 172.21.15.143:0/1072602023 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7f134c01b3b0 con 0x7f135c06f200 2023-09-29T23:34:51.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.853+0000 7f13517fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:51.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.853+0000 7f13517fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:51.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.853+0000 7f13517fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:51.853 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.853+0000 7f13517fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:51.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.853+0000 7f13517fa700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f134008e8b0 0x7f1340090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:51.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.853+0000 7f13517fa700 1 -- 172.21.15.143:0/1072602023 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7f134c0b40b0 con 0x7f135c06f200 2023-09-29T23:34:51.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.853+0000 7f1361524700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f134008e8b0 0x7f1340090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:51.854 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.853+0000 7f1361524700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f134008e8b0 0x7f1340090d60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1358009fd0 tx=0x7f1358017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:51.862 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:51.861+0000 7f13517fa700 1 -- 172.21.15.143:0/1072602023 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7f134c084a70 con 0x7f135c06f200 2023-09-29T23:34:52.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 -- 172.21.15.143:0/1072602023 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f135c140840 con 0x7f135c06f200 2023-09-29T23:34:52.060 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f13517fa700 1 -- 172.21.15.143:0/1072602023 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+227 (secure 0 0 0) 0x7f134c017070 con 0x7f135c06f200 2023-09-29T23:34:52.060 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:52.061 INFO:teuthology.orchestra.run.smithi143.stdout:{"status":"HEALTH_ERR","checks":{"MGR_MODULE_ERROR":{"severity":"HEALTH_ERR","summary":{"message":"Module 'pg_autoscaler' has failed: 'PgAutoscaler' object has no attribute 'noautoscale'","count":1},"muted":false}},"mutes":[]} 2023-09-29T23:34:52.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 10 mgrc shutdown 2023-09-29T23:34:52.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 -- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f134008e8b0 msgr2=0x7f1340090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f134008e8b0 0x7f1340090d60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1358009fd0 tx=0x7f1358017000 comp rx=0 tx=0).stop 2023-09-29T23:34:52.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 -- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06f200 msgr2=0x7f135c0a8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06f200 0x7f135c0a8070 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c5c0 tx=0x7f134c007ec0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 -- 172.21.15.143:0/1072602023 shutdown_connections 2023-09-29T23:34:52.063 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7f134008e8b0 0x7f1340090d60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f135c13fbb0 0x7f135c0a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7f135c06f200 0x7f135c0a8070 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 --2- 172.21.15.143:0/1072602023 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f135c06ee30 0x7f135c0a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 -- 172.21.15.143:0/1072602023 >> 172.21.15.143:0/1072602023 conn(0x7f135c0b2ca0 msgr2=0x7f135c066c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:52.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 -- 172.21.15.143:0/1072602023 shutdown_connections 2023-09-29T23:34:52.064 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.061+0000 7f1362526700 1 -- 172.21.15.143:0/1072602023 wait complete. 2023-09-29T23:34:52.075 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-09-29T23:34:52.075 DEBUG:teuthology.orchestra.run.smithi143:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-29T23:34:52.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.177+0000 7fdeaedf4700 1 Processor -- start 2023-09-29T23:34:52.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.177+0000 7fdeaedf4700 1 -- start start 2023-09-29T23:34:52.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea806efd0 0x7fdea80a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdea809fc40 0x7fdea80a0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea80a1f10 con 0x7fdea80a4260 2023-09-29T23:34:52.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea80a0fe0 con 0x7fdea809fc40 2023-09-29T23:34:52.180 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80a58f0 con 0x7fdea806efd0 2023-09-29T23:34:52.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaddf2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea806efd0 0x7fdea80a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaddf2700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea806efd0 0x7fdea80a19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:42564/0 (socket says 172.21.15.143:42564) 2023-09-29T23:34:52.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaddf2700 1 -- 172.21.15.143:0/1059337439 learned_addr learned my addr 172.21.15.143:0/1059337439 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:52.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdead5f1700 1 --2- 172.21.15.143:0/1059337439 >> v2:172.21.15.156:3300/0 conn(0x7fdea809fc40 0x7fdea80a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaddf2700 1 -- 172.21.15.143:0/1059337439 >> v2:172.21.15.156:3300/0 conn(0x7fdea809fc40 msgr2=0x7fdea80a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:52.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaddf2700 1 --2- 172.21.15.143:0/1059337439 >> v2:172.21.15.156:3300/0 conn(0x7fdea809fc40 0x7fdea80a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeacdf0700 1 -- 172.21.15.143:0/1059337439 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595392349 0 0) 0x7fdea80a1f10 con 0x7fdea80a4260 2023-09-29T23:34:52.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaddf2700 1 -- 172.21.15.143:0/1059337439 >> v1:172.21.15.156:6789/0 conn(0x7fdea80a4260 legacy=0x7fdea80a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.182 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaddf2700 1 -- 172.21.15.143:0/1059337439 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80a23a0 con 0x7fdea806efd0 2023-09-29T23:34:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaddf2700 1 --2- 172.21.15.143:0/1059337439 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea806efd0 0x7fdea80a19a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdea40099b0 tx=0x7fdea4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=881647753d03ddf4 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeacdf0700 1 -- 172.21.15.143:0/1059337439 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea4009ea0 con 0x7fdea806efd0 2023-09-29T23:34:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeacdf0700 1 -- 172.21.15.143:0/1059337439 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdea400b860 con 0x7fdea806efd0 2023-09-29T23:34:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeacdf0700 1 -- 172.21.15.143:0/1059337439 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdea400bc00 con 0x7fdea806efd0 2023-09-29T23:34:52.183 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1059337439 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea806efd0 msgr2=0x7fdea80a19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 --2- 172.21.15.143:0/1059337439 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea806efd0 0x7fdea80a19a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdea40099b0 tx=0x7fdea4009250 comp rx=0 tx=0).stop 2023-09-29T23:34:52.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1059337439 shutdown_connections 2023-09-29T23:34:52.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 --2- 172.21.15.143:0/1059337439 >> v2:172.21.15.156:3300/0 conn(0x7fdea809fc40 0x7fdea80a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 --2- 172.21.15.143:0/1059337439 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea806efd0 0x7fdea80a19a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1059337439 >> 172.21.15.143:0/1059337439 conn(0x7fdea80b2ca0 msgr2=0x7fdea80b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:52.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1059337439 shutdown_connections 2023-09-29T23:34:52.184 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.181+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1059337439 wait complete. 2023-09-29T23:34:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdeaedf4700 1 Processor -- start 2023-09-29T23:34:52.185 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdeaedf4700 1 -- start start 2023-09-29T23:34:52.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdeaedf4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea806efd0 0x7fdea8140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdeaedf4700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea809fc40 0x7fdea8097b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.187 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdeaedf4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdea80a4260 0x7fdea80980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdeaedf4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea8098610 con 0x7fdea80a4260 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdeaedf4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea806b380 con 0x7fdea806efd0 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdeaedf4700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea8098820 con 0x7fdea809fc40 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdead5f1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea809fc40 0x7fdea8097b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdead5f1700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea809fc40 0x7fdea8097b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:42580/0 (socket says 172.21.15.143:42580) 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdeae5f3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdea80a4260 0x7fdea80980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdead5f1700 1 -- 172.21.15.143:0/1330621900 learned_addr learned my addr 172.21.15.143:0/1330621900 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdeaddf2700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea806efd0 0x7fdea8140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdead5f1700 1 -- 172.21.15.143:0/1330621900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea806efd0 msgr2=0x7fdea8140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdead5f1700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea806efd0 0x7fdea8140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdead5f1700 1 -- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdea80a4260 msgr2=0x7fdea80980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.188 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdead5f1700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdea80a4260 0x7fdea80980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.185+0000 7fdead5f1700 1 -- 172.21.15.143:0/1330621900 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea40094f0 con 0x7fdea809fc40 2023-09-29T23:34:52.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.189+0000 7fdead5f1700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea809fc40 0x7fdea8097b90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde9800ec50 tx=0x7fde980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:52.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.189+0000 7fde9dffb700 1 -- 172.21.15.143:0/1330621900 <== mon.2 v2:172.21.15.143:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9800c850 con 0x7fdea809fc40 2023-09-29T23:34:52.189 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.189+0000 7fde9dffb700 1 -- 172.21.15.143:0/1330621900 <== mon.2 v2:172.21.15.143:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9800c9d0 con 0x7fdea809fc40 2023-09-29T23:34:52.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.189+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1330621900 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea8098ae0 con 0x7fdea809fc40 2023-09-29T23:34:52.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.189+0000 7fde9dffb700 1 -- 172.21.15.143:0/1330621900 <== mon.2 v2:172.21.15.143:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9800cd90 con 0x7fdea809fc40 2023-09-29T23:34:52.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.189+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1330621900 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea8141000 con 0x7fdea809fc40 2023-09-29T23:34:52.190 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.189+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1330621900 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea805ecb0 con 0x7fdea809fc40 2023-09-29T23:34:52.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.189+0000 7fde9dffb700 1 -- 172.21.15.143:0/1330621900 <== mon.2 v2:172.21.15.143:3301/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fde98013680 con 0x7fdea809fc40 2023-09-29T23:34:52.191 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.189+0000 7fde9dffb700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.193+0000 7fde9dffb700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.193+0000 7fde9dffb700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:52.192 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.193+0000 7fde9dffb700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.193+0000 7fde9dffb700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fde8c08e8b0 0x7fde8c090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.193+0000 7fdeaddf2700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fde8c08e8b0 0x7fde8c090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.193+0000 7fde9dffb700 1 -- 172.21.15.143:0/1330621900 <== mon.2 v2:172.21.15.143:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fde980b4e70 con 0x7fdea809fc40 2023-09-29T23:34:52.193 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.193+0000 7fdeaddf2700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fde8c08e8b0 0x7fde8c090d60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdea40099b0 tx=0x7fdea4017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:52.200 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.201+0000 7fde9dffb700 1 -- 172.21.15.143:0/1330621900 <== mon.2 v2:172.21.15.143:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fde98085830 con 0x7fdea809fc40 2023-09-29T23:34:52.358 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.357+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1330621900 --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdea80a5af0 con 0x7fdea809fc40 2023-09-29T23:34:52.360 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fde9dffb700 1 -- 172.21.15.143:0/1330621900 <== mon.2 v2:172.21.15.143:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+12393 (secure 0 0 0) 0x7fde98017020 con 0x7fdea809fc40 2023-09-29T23:34:52.360 INFO:teuthology.orchestra.run.smithi143.stdout: 2023-09-29T23:34:52.360 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":21,"fsid":"d0aefd66-f5df-4d76-82e3-362118c32b1d","created":"2023-09-29T23:29:18.315936+0000","modified":"2023-09-29T23:30:24.140140+0000","last_up_change":"2023-09-29T23:29:27.340059+0000","last_in_change":"2023-09-29T23:29:19.370491+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-29T23:29:28.520729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-29T23:29:33.891889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-29T23:29:37.927418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-29T23:29:38.594861+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"c268b844-dbdc-482b-965f-f38c2a1da791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":27741},{"type":"v1","addr":"172.21.15.143:6801","nonce":27741}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":27741},{"type":"v1","addr":"172.21.15.143:6803","nonce":27741}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":27741},{"type":"v1","addr":"172.21.15.143:6807","nonce":27741}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":27741},{"type":"v1","addr":"172.21.15.143:6805","nonce":27741}]},"public_addr":"172.21.15.143:6801/27741","cluster_addr":"172.21.15.143:6803/27741","heartbeat_back_addr":"172.21.15.143:6807/27741","heartbeat_front_addr":"172.21.15.143:6805/27741","state":["exists","up"]},{"osd":1,"uuid":"ab4ecead-b2d8-4c5f-849e-fc69aee684ed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":30036},{"type":"v1","addr":"172.21.15.156:6817","nonce":30036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":30036},{"type":"v1","addr":"172.21.15.156:6819","nonce":30036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":30036},{"type":"v1","addr":"172.21.15.156:6823","nonce":30036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":30036},{"type":"v1","addr":"172.21.15.156:6821","nonce":30036}]},"public_addr":"172.21.15.156:6817/30036","cluster_addr":"172.21.15.156:6819/30036","heartbeat_back_addr":"172.21.15.156:6823/30036","heartbeat_front_addr":"172.21.15.156:6821/30036","state":["exists","up"]},{"osd":2,"uuid":"907c543a-0ea3-4163-84a6-f50f5ed849fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":30035},{"type":"v1","addr":"172.21.15.156:6809","nonce":30035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":30035},{"type":"v1","addr":"172.21.15.156:6811","nonce":30035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":30035},{"type":"v1","addr":"172.21.15.156:6815","nonce":30035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":30035},{"type":"v1","addr":"172.21.15.156:6813","nonce":30035}]},"public_addr":"172.21.15.156:6809/30035","cluster_addr":"172.21.15.156:6811/30035","heartbeat_back_addr":"172.21.15.156:6815/30035","heartbeat_front_addr":"172.21.15.156:6813/30035","state":["exists","up"]},{"osd":3,"uuid":"bf5a122c-0873-4890-8845-702c4fc3325b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":30037},{"type":"v1","addr":"172.21.15.156:6801","nonce":30037}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":30037},{"type":"v1","addr":"172.21.15.156:6803","nonce":30037}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":30037},{"type":"v1","addr":"172.21.15.156:6807","nonce":30037}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":30037},{"type":"v1","addr":"172.21.15.156:6805","nonce":30037}]},"public_addr":"172.21.15.156:6801/30037","cluster_addr":"172.21.15.156:6803/30037","heartbeat_back_addr":"172.21.15.156:6807/30037","heartbeat_front_addr":"172.21.15.156:6805/30037","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.781977+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.943955+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.806513+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-09-29T23:29:24.624301+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.5","mappings":[{"from":3,"to":2}]},{"pgid":"4.a","mappings":[{"from":3,"to":0}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-29T23:34:52.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 10 mgrc shutdown 2023-09-29T23:34:52.363 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fde8c08e8b0 msgr2=0x7fde8c090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fde8c08e8b0 0x7fde8c090d60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdea40099b0 tx=0x7fdea4017000 comp rx=0 tx=0).stop 2023-09-29T23:34:52.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea809fc40 msgr2=0x7fdea8097b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea809fc40 0x7fdea8097b90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fde9800ec50 tx=0x7fde980091a0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1330621900 shutdown_connections 2023-09-29T23:34:52.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fde8c08e8b0 0x7fde8c090d60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.364 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdea80a4260 0x7fdea80980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fdea809fc40 0x7fdea8097b90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 --2- 172.21.15.143:0/1330621900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdea806efd0 0x7fdea8140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1330621900 >> 172.21.15.143:0/1330621900 conn(0x7fdea80b2ca0 msgr2=0x7fdea80a46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:52.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1330621900 shutdown_connections 2023-09-29T23:34:52.365 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.361+0000 7fdeaedf4700 1 -- 172.21.15.143:0/1330621900 wait complete. 2023-09-29T23:34:52.376 DEBUG:teuthology.orchestra.run.smithi143:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-09-29T23:34:52.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.481+0000 7fd2de59c700 1 Processor -- start 2023-09-29T23:34:52.482 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.481+0000 7fd2de59c700 1 -- start start 2023-09-29T23:34:52.483 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.481+0000 7fd2de59c700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2d813c820 0x7fd2d80a2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.481+0000 7fd2de59c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d80b33c0 0x7fd2d8147570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.481+0000 7fd2de59c700 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80a2770 con 0x7fd2d80a5250 2023-09-29T23:34:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2de59c700 1 -- --> v2:172.21.15.143:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d80a1600 con 0x7fd2d813c820 2023-09-29T23:34:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2de59c700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80a57c0 con 0x7fd2d80b33c0 2023-09-29T23:34:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2dd59a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2d813c820 0x7fd2d80a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2ddd9b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d80b33c0 0x7fd2d8147570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2ddd9b700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d80b33c0 0x7fd2d8147570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3301/0 says I am v2:172.21.15.143:42596/0 (socket says 172.21.15.143:42596) 2023-09-29T23:34:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2ddd9b700 1 -- 172.21.15.143:0/1016969017 learned_addr learned my addr 172.21.15.143:0/1016969017 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:52.484 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2dd59a700 1 --2- >> v2:172.21.15.143:3300/0 conn(0x7fd2d813c820 0x7fd2d80a2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43488/0 (socket says 172.21.15.143:43488) 2023-09-29T23:34:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2dd59a700 1 -- 172.21.15.143:0/1016969017 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d80b33c0 msgr2=0x7fd2d8147570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-29T23:34:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2dd59a700 1 --2- 172.21.15.143:0/1016969017 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d80b33c0 0x7fd2d8147570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2dd59a700 1 -- 172.21.15.143:0/1016969017 >> v1:172.21.15.156:6789/0 conn(0x7fd2d80a5250 legacy=0x7fd2d80a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2dd59a700 1 -- 172.21.15.143:0/1016969017 --> v2:172.21.15.143:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d80a5a80 con 0x7fd2d813c820 2023-09-29T23:34:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2ddd9b700 1 --2- 172.21.15.143:0/1016969017 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d80b33c0 0x7fd2d8147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-29T23:34:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2dd59a700 1 --2- 172.21.15.143:0/1016969017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d813c820 0x7fd2d80a2230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800dd00 tx=0x7fd2c800d660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c9f51e6cd33843 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2cffff700 1 -- 172.21.15.143:0/1016969017 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c8012070 con 0x7fd2d813c820 2023-09-29T23:34:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2cffff700 1 -- 172.21.15.143:0/1016969017 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c800de60 con 0x7fd2d813c820 2023-09-29T23:34:52.487 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2cffff700 1 -- 172.21.15.143:0/1016969017 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c800fd90 con 0x7fd2d813c820 2023-09-29T23:34:52.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2de59c700 1 -- 172.21.15.143:0/1016969017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d813c820 msgr2=0x7fd2d80a2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2de59c700 1 --2- 172.21.15.143:0/1016969017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d813c820 0x7fd2d80a2230 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800dd00 tx=0x7fd2c800d660 comp rx=0 tx=0).stop 2023-09-29T23:34:52.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2de59c700 1 -- 172.21.15.143:0/1016969017 shutdown_connections 2023-09-29T23:34:52.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2de59c700 1 --2- 172.21.15.143:0/1016969017 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d80b33c0 0x7fd2d8147570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2de59c700 1 --2- 172.21.15.143:0/1016969017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d813c820 0x7fd2d80a2230 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2de59c700 1 -- 172.21.15.143:0/1016969017 >> 172.21.15.143:0/1016969017 conn(0x7fd2d8062c70 msgr2=0x7fd2d8063070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:52.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2de59c700 1 -- 172.21.15.143:0/1016969017 shutdown_connections 2023-09-29T23:34:52.488 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.485+0000 7fd2de59c700 1 -- 172.21.15.143:0/1016969017 wait complete. 2023-09-29T23:34:52.490 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2de59c700 1 Processor -- start 2023-09-29T23:34:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2de59c700 1 -- start start 2023-09-29T23:34:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2de59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d80a5250 0x7fd2d813e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2de59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b33c0 0x7fd2d8136340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2de59c700 1 --2- >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d813c820 0x7fd2d814c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2de59c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d813e570 con 0x7fd2d80b33c0 2023-09-29T23:34:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2de59c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d814bda0 con 0x7fd2d80a5250 2023-09-29T23:34:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2de59c700 1 -- --> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8135070 con 0x7fd2d813c820 2023-09-29T23:34:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2dcd99700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b33c0 0x7fd2d8136340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2dcd99700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b33c0 0x7fd2d8136340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:43492/0 (socket says 172.21.15.143:43492) 2023-09-29T23:34:52.491 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2dcd99700 1 -- 172.21.15.143:0/374041651 learned_addr learned my addr 172.21.15.143:0/374041651 (peer_addr_for_me v2:172.21.15.143:0/0) 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2ddd9b700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d813c820 0x7fd2d814c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2dd59a700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d80a5250 0x7fd2d813e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2dcd99700 1 -- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d813c820 msgr2=0x7fd2d814c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2dcd99700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d813c820 0x7fd2d814c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2dcd99700 1 -- 172.21.15.143:0/374041651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d80a5250 msgr2=0x7fd2d813e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2dcd99700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d80a5250 0x7fd2d813e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2dcd99700 1 -- 172.21.15.143:0/374041651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d0009ae0 con 0x7fd2d80b33c0 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2dd59a700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d80a5250 0x7fd2d813e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2ddd9b700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d813c820 0x7fd2d814c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2dcd99700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b33c0 0x7fd2d8136340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400c590 tx=0x7fd2d400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2cd7fa700 1 -- 172.21.15.143:0/374041651 <== mon.0 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4011590 con 0x7fd2d80b33c0 2023-09-29T23:34:52.492 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2cd7fa700 1 -- 172.21.15.143:0/374041651 <== mon.0 v2:172.21.15.143:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d40116f0 con 0x7fd2d80b33c0 2023-09-29T23:34:52.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2de59c700 1 -- 172.21.15.143:0/374041651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c800d8c0 con 0x7fd2d80b33c0 2023-09-29T23:34:52.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2cd7fa700 1 -- 172.21.15.143:0/374041651 <== mon.0 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d4011ab0 con 0x7fd2d80b33c0 2023-09-29T23:34:52.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.489+0000 7fd2de59c700 1 -- 172.21.15.143:0/374041651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d814ba50 con 0x7fd2d80b33c0 2023-09-29T23:34:52.493 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.493+0000 7fd2de59c700 1 -- 172.21.15.143:0/374041651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d814bf10 con 0x7fd2d80b33c0 2023-09-29T23:34:52.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.493+0000 7fd2cd7fa700 1 -- 172.21.15.143:0/374041651 <== mon.0 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 5) v1 ==== 119739+0+0 (secure 0 0 0) 0x7fd2d401b3b0 con 0x7fd2d80b33c0 2023-09-29T23:34:52.494 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.493+0000 7fd2cd7fa700 20 mgrc handle_mgr_map mgrmap(e 5) v1 2023-09-29T23:34:52.495 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.497+0000 7fd2cd7fa700 4 mgrc handle_mgr_map Got map version 5 2023-09-29T23:34:52.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.497+0000 7fd2cd7fa700 4 mgrc handle_mgr_map Active mgr is now [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:52.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.497+0000 7fd2cd7fa700 4 mgrc reconnect Starting new session with [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] 2023-09-29T23:34:52.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.497+0000 7fd2cd7fa700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd2bc08e8b0 0x7fd2bc090d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-29T23:34:52.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.497+0000 7fd2dd59a700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd2bc08e8b0 0x7fd2bc090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-29T23:34:52.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.497+0000 7fd2cd7fa700 1 -- 172.21.15.143:0/374041651 <== mon.0 v2:172.21.15.143:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4446+0+0 (secure 0 0 0) 0x7fd2d40b40a0 con 0x7fd2d80b33c0 2023-09-29T23:34:52.496 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.497+0000 7fd2dd59a700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd2bc08e8b0 0x7fd2bc090d60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800dd00 tx=0x7fd2c801b000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-29T23:34:52.498 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.497+0000 7fd2cd7fa700 1 -- 172.21.15.143:0/374041651 <== mon.0 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+166856 (secure 0 0 0) 0x7fd2d4084a50 con 0x7fd2d80b33c0 2023-09-29T23:34:52.658 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.657+0000 7fd2de59c700 1 -- 172.21.15.143:0/374041651 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fd2d80a2720 con 0x7fd2d80b33c0 2023-09-29T23:34:52.667 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.665+0000 7fd2cd7fa700 1 -- 172.21.15.143:0/374041651 <== mon.0 v2:172.21.15.143:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd2d4084a50 con 0x7fd2d80b33c0 2023-09-29T23:34:52.668 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.669+0000 7fd2cd7fa700 1 -- 172.21.15.143:0/374041651 <== mon.0 v2:172.21.15.143:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) v1 ==== 123+0+0 (secure 0 0 0) 0x7fd2d4017070 con 0x7fd2d80b33c0 2023-09-29T23:34:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.669+0000 7fd2de59c700 10 mgrc shutdown 2023-09-29T23:34:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.669+0000 7fd2de59c700 1 -- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd2bc08e8b0 msgr2=0x7fd2bc090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.669+0000 7fd2de59c700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd2bc08e8b0 0x7fd2bc090d60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd2c800dd00 tx=0x7fd2c801b000 comp rx=0 tx=0).stop 2023-09-29T23:34:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.669+0000 7fd2de59c700 1 -- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b33c0 msgr2=0x7fd2d8136340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-29T23:34:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.669+0000 7fd2de59c700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b33c0 0x7fd2d8136340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd2d400c590 tx=0x7fd2d400c950 comp rx=0 tx=0).stop 2023-09-29T23:34:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.669+0000 7fd2de59c700 1 -- 172.21.15.143:0/374041651 shutdown_connections 2023-09-29T23:34:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.669+0000 7fd2de59c700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:6808/27602,v1:172.21.15.143:6809/27602] conn(0x7fd2bc08e8b0 0x7fd2bc090d60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.672 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.669+0000 7fd2de59c700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:3301/0,v1:172.21.15.143:6790/0] conn(0x7fd2d813c820 0x7fd2d814c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.673+0000 7fd2de59c700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd2d80b33c0 0x7fd2d8136340 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.673+0000 7fd2de59c700 1 --2- 172.21.15.143:0/374041651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2d80a5250 0x7fd2d813e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-29T23:34:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.673+0000 7fd2de59c700 1 -- 172.21.15.143:0/374041651 >> 172.21.15.143:0/374041651 conn(0x7fd2d8062c70 msgr2=0x7fd2d80a1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-29T23:34:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.673+0000 7fd2de59c700 1 -- 172.21.15.143:0/374041651 shutdown_connections 2023-09-29T23:34:52.673 INFO:teuthology.orchestra.run.smithi143.stderr:2023-09-29T23:34:52.673+0000 7fd2de59c700 1 -- 172.21.15.143:0/374041651 wait complete. 2023-09-29T23:34:52.683 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1895, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1472, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph_manager.py", line 3189, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-09-29T23:34:52.684 INFO:teuthology.misc:Shutting down mds daemons... 2023-09-29T23:34:52.684 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-29T23:34:52.685 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:52.695 INFO:tasks.ceph.mds.a:Stopped 2023-09-29T23:34:52.696 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-09-29T23:34:52.696 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:52.707 INFO:tasks.ceph.mds.c:Stopped 2023-09-29T23:34:52.707 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-09-29T23:34:52.708 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:52.724 INFO:tasks.ceph.mds.b:Stopped 2023-09-29T23:34:52.724 INFO:teuthology.misc:Shutting down osd daemons... 2023-09-29T23:34:52.724 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-29T23:34:52.724 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:52.819 INFO:tasks.ceph.osd.0:Stopped 2023-09-29T23:34:52.819 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-29T23:34:52.819 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:52.888 INFO:tasks.ceph.osd.1:Stopped 2023-09-29T23:34:52.888 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-29T23:34:52.889 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:56.461 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:34:56.464 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:34:57.117 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-09-29T23:34:58.991 INFO:tasks.ceph.osd.2:Stopped 2023-09-29T23:34:58.991 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-29T23:34:58.991 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:59.051 INFO:tasks.ceph.osd.3:Stopped 2023-09-29T23:34:59.051 INFO:teuthology.misc:Shutting down mgr daemons... 2023-09-29T23:34:59.051 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-29T23:34:59.051 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:59.081 INFO:tasks.ceph.mgr.x:Stopped 2023-09-29T23:34:59.081 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-09-29T23:34:59.081 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:59.111 INFO:tasks.ceph.mgr.y:Stopped 2023-09-29T23:34:59.112 DEBUG:tasks.ceph.mgr.z:waiting for process to exit 2023-09-29T23:34:59.112 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:59.142 INFO:tasks.ceph.mgr.z:Stopped 2023-09-29T23:34:59.142 INFO:teuthology.misc:Shutting down mon daemons... 2023-09-29T23:34:59.143 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-29T23:34:59.143 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:59.162 INFO:tasks.ceph.mon.a:Stopped 2023-09-29T23:34:59.163 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-29T23:34:59.163 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:59.220 INFO:tasks.ceph.mon.c:Stopped 2023-09-29T23:34:59.221 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-29T23:34:59.221 INFO:teuthology.orchestra.run:waiting for 300 2023-09-29T23:34:59.242 INFO:tasks.ceph.mon.b:Stopped 2023-09-29T23:34:59.242 INFO:tasks.ceph:Checking cluster log for badness... 2023-09-29T23:34:59.242 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v 'replacing it with standby' | egrep -v 'No standby daemons available' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2023-09-29T23:34:59.261 INFO:teuthology.orchestra.run.smithi143.stdout:1696030162.4785848 mgr.x (mgr.4101) 1 : cluster [ERR] Unhandled exception from module 'pg_autoscaler' while running on mgr.x: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-29T23:34:59.262 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-09-29T23:34:59.262 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v 'replacing it with standby' | egrep -v 'No standby daemons available' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2023-09-29T23:34:59.279 DEBUG:teuthology.orchestra.run.smithi143:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v 'replacing it with standby' | egrep -v 'No standby daemons available' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2023-09-29T23:34:59.340 INFO:teuthology.orchestra.run.smithi143.stdout:1696030162.4785848 mgr.x (mgr.4101) 1 : cluster [ERR] Unhandled exception from module 'pg_autoscaler' while running on mgr.x: 'PgAutoscaler' object has no attribute 'noautoscale' 2023-09-29T23:34:59.341 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T23:34:59.341 DEBUG:teuthology.orchestra.run.smithi143:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-09-29T23:34:59.656 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi156.front.sepia.ceph.com 2023-09-29T23:34:59.656 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-09-29T23:34:59.970 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi156.front.sepia.ceph.com 2023-09-29T23:34:59.971 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-09-29T23:35:00.273 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi156.front.sepia.ceph.com 2023-09-29T23:35:00.274 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-09-29T23:35:00.511 INFO:tasks.ceph:Archiving mon data... 2023-09-29T23:35:00.512 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568/data/mon.a.tgz 2023-09-29T23:35:00.513 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2023-09-29T23:35:00.520 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.BSaK5opIq3 2023-09-29T23:35:00.520 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.BSaK5opIq3 2023-09-29T23:35:00.642 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.BSaK5opIq3 2023-09-29T23:35:00.666 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.BSaK5opIq3 is 311KB 2023-09-29T23:35:00.729 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.BSaK5opIq3 2023-09-29T23:35:00.736 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568/data/mon.c.tgz 2023-09-29T23:35:00.736 DEBUG:teuthology.orchestra.run.smithi143:> mktemp 2023-09-29T23:35:00.743 INFO:teuthology.orchestra.run.smithi143.stdout:/tmp/tmp.vP62sOm7SQ 2023-09-29T23:35:00.743 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.vP62sOm7SQ 2023-09-29T23:35:00.845 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 0666 /tmp/tmp.vP62sOm7SQ 2023-09-29T23:35:00.872 DEBUG:teuthology.orchestra.remote:smithi143:/tmp/tmp.vP62sOm7SQ is 433KB 2023-09-29T23:35:00.890 DEBUG:teuthology.orchestra.run.smithi143:> rm -fr /tmp/tmp.vP62sOm7SQ 2023-09-29T23:35:00.896 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568/data/mon.b.tgz 2023-09-29T23:35:00.896 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2023-09-29T23:35:00.903 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.d3VqJaON8h 2023-09-29T23:35:00.903 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.d3VqJaON8h 2023-09-29T23:35:01.008 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.d3VqJaON8h 2023-09-29T23:35:01.024 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.d3VqJaON8h is 433KB 2023-09-29T23:35:01.088 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.d3VqJaON8h 2023-09-29T23:35:01.095 INFO:tasks.ceph:Cleaning ceph cluster... 2023-09-29T23:35:01.095 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-29T23:35:01.097 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-29T23:35:01.113 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-09-29T23:35:01.113 DEBUG:teuthology.orchestra.run.smithi143:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-29T23:35:01.157 DEBUG:teuthology.orchestra.run.smithi156:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-29T23:35:01.182 INFO:teuthology.orchestra.run.smithi143.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-29T23:35:01.187 INFO:teuthology.orchestra.run.smithi156.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-29T23:35:01.194 INFO:tasks.ceph:Archiving crash dumps... 2023-09-29T23:35:01.195 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568/remote/smithi143/crash 2023-09-29T23:35:01.196 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-29T23:35:01.214 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568/remote/smithi156/crash 2023-09-29T23:35:01.215 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-29T23:35:01.252 INFO:tasks.ceph:Shutting down logrotate 2023-09-29T23:35:01.252 DEBUG:teuthology.orchestra.run.smithi143:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:35:01.255 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-09-29T23:35:01.306 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-09-29T23:35:01.357 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-09-29T23:35:01.371 INFO:tasks.ceph:Compressing logs... 2023-09-29T23:35:01.371 DEBUG:teuthology.orchestra.run.smithi143:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-29T23:35:01.373 DEBUG:teuthology.orchestra.run.smithi156:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-29T23:35:02.119 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-09-29T23:35:04.003 INFO:tasks.ceph:Archiving logs... 2023-09-29T23:35:04.003 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568/remote/smithi143/log 2023-09-29T23:35:04.004 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-29T23:35:04.271 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568/remote/smithi156/log 2023-09-29T23:35:04.272 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-29T23:35:04.623 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1895, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1472, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph_manager.py", line 3189, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-09-29T23:35:04.701 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=4a49c396c37e46f8bec0108790c45777 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1895, in task healthy(ctx=ctx, config=dict(cluster=config['cluster'])) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph.py", line 1472, in healthy manager.wait_until_healthy(timeout=300) File "/home/teuthworker/src/github.com_ceph_ceph-c_93dce5e17918f8aca65993a7a2dc40233a47b481/qa/tasks/ceph_manager.py", line 3189, in wait_until_healthy assert time.time() - start < timeout, \ AssertionError: timeout expired in wait_until_healthy 2023-09-29T23:35:04.705 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-09-29T23:35:04.715 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-29T23:35:04.730 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-29T23:35:04.731 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-29T23:35:04.751 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-29T23:35:04.815 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-29T23:35:04.824 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-29T23:35:04.824 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-29T23:35:04.827 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-29T23:35:04.848 INFO:teuthology.orchestra.run.smithi143.stdout: remote refid st t when poll reach delay offset jitter 2023-09-29T23:35:04.848 INFO:teuthology.orchestra.run.smithi143.stdout:============================================================================== 2023-09-29T23:35:04.848 INFO:teuthology.orchestra.run.smithi143.stdout:*hv01.front.sepi 44.4.53.2 2 u 30 64 377 0.067 0.367 0.760 2023-09-29T23:35:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: hv02.front.sepi 63.231.80.2 3 u 33 64 377 0.084 0.546 0.735 2023-09-29T23:35:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: hv03.front.sepi 74.6.168.72 3 u 27 64 377 0.107 -0.013 1.412 2023-09-29T23:35:04.849 INFO:teuthology.orchestra.run.smithi143.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-29T23:35:04.850 INFO:teuthology.orchestra.run.smithi156.stdout: remote refid st t when poll reach delay offset jitter 2023-09-29T23:35:04.850 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================== 2023-09-29T23:35:04.850 INFO:teuthology.orchestra.run.smithi156.stdout:*hv01.front.sepi 44.4.53.2 2 u 28 64 377 0.093 0.591 0.599 2023-09-29T23:35:04.850 INFO:teuthology.orchestra.run.smithi156.stdout:+hv02.front.sepi 63.231.80.2 3 u 32 64 377 0.060 0.827 0.522 2023-09-29T23:35:04.851 INFO:teuthology.orchestra.run.smithi156.stdout:+hv03.front.sepi 74.6.168.72 3 u 23 64 377 0.084 0.169 0.755 2023-09-29T23:35:04.851 INFO:teuthology.orchestra.run.smithi156.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2023-09-29T23:35:04.852 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-29T23:35:04.860 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-29T23:35:04.861 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-29T23:35:04.869 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-29T23:35:04.878 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-29T23:35:04.890 INFO:teuthology.task.internal:Duration was 842.428182 seconds 2023-09-29T23:35:04.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-29T23:35:04.900 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-29T23:35:04.900 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-29T23:35:04.902 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-29T23:35:04.952 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-29T23:35:04.953 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2023-09-29T23:35:04.953 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-29T23:35:04.967 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2023-09-29T23:35:04.967 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-29T23:35:04.981 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-29T23:35:04.981 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-29T23:35:04.983 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-29T23:35:05.010 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-29T23:35:05.010 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-29T23:35:05.014 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-29T23:35:05.114 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-29T23:35:05.123 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-29T23:35:05.124 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-29T23:35:05.138 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-29T23:35:05.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-29T23:35:05.164 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-29T23:35:05.166 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-29T23:35:05.179 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2023-09-29T23:35:05.179 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2023-09-29T23:35:05.212 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-29T23:35:05.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T23:35:05.261 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-29T23:35:05.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-29T23:35:05.267 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-29T23:35:05.277 INFO:teuthology.task.internal:Transferring archived files... 2023-09-29T23:35:05.277 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568/remote/smithi143 2023-09-29T23:35:05.278 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-29T23:35:05.301 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568/remote/smithi156 2023-09-29T23:35:05.301 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-29T23:35:05.322 INFO:teuthology.task.internal:Removing archive directory... 2023-09-29T23:35:05.322 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-29T23:35:05.341 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2023-09-29T23:35:05.368 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-29T23:35:05.378 INFO:teuthology.task.internal:Not uploading archives. 2023-09-29T23:35:05.379 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-29T23:35:05.387 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-29T23:35:05.387 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-29T23:35:05.390 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-29T23:35:05.395 INFO:teuthology.orchestra.run.smithi143.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 29 23:35 /home/ubuntu/cephtest 2023-09-29T23:35:05.398 INFO:teuthology.orchestra.run.smithi156.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 29 23:35 /home/ubuntu/cephtest 2023-09-29T23:35:05.400 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-29T23:35:05.410 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-29T23:35:05.433 INFO:teuthology.nuke:Checking targets against current locks 2023-09-29T23:35:05.461 DEBUG:teuthology.nuke:shortname: smithi143 2023-09-29T23:35:05.461 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-29T23:35:05.484 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-29 23:11:53.389423', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-09-29T23:35:05.488 DEBUG:teuthology.nuke:shortname: smithi156 2023-09-29T23:35:05.488 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-29T23:35:05.508 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-09-29_19:38:56-rados-wip-yuri4-testing-2023-09-29-0928-quincy-distro-default-smithi/7406568', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-29 23:11:53.390738', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='} 2023-09-29T23:35:05.538 INFO:teuthology.orchestra.console.smithi143:Power off 2023-09-29T23:35:05.539 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-29T23:35:05.562 DEBUG:teuthology.orchestra.console.smithi143:power off output: Chassis Power Control: Down/Off 2023-09-29T23:35:05.562 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T23:35:05.580 INFO:teuthology.orchestra.console.smithi156:Power off 2023-09-29T23:35:05.580 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-29T23:35:05.590 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2023-09-29T23:35:05.605 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2023-09-29T23:35:05.605 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T23:35:05.629 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-09-29T23:35:09.591 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T23:35:09.631 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T23:35:09.701 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2023-09-29T23:35:09.741 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-09-29T23:35:13.704 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T23:35:13.742 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T23:35:13.814 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is on 2023-09-29T23:35:13.852 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-09-29T23:35:17.815 DEBUG:teuthology.orchestra.console.smithi143:pexpect command: ipmitool -H smithi143.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T23:35:17.853 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-29T23:35:17.925 DEBUG:teuthology.orchestra.console.smithi143:check power output: Chassis Power is off 2023-09-29T23:35:17.964 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2023-09-29T23:35:18.026 INFO:teuthology.orchestra.console.smithi143:Power off completed 2023-09-29T23:35:18.065 INFO:teuthology.orchestra.console.smithi156:Power off completed 2023-09-29T23:35:18.170 INFO:teuthology.run:Summary data: description: rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/failover} duration: 842.4281823635101 failure_reason: '"1696030162.4785848 mgr.x (mgr.4101) 1 : cluster [ERR] Unhandled exception from module ''pg_autoscaler'' while running on mgr.x: ''PgAutoscaler'' object has no attribute ''noautoscale''" in cluster log' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=4a49c396c37e46f8bec0108790c45777 status: fail success: false 2023-09-29T23:35:18.170 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-29T23:35:18.356 INFO:teuthology.run:FAIL